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ABSTRACT 


In  August  2012,  Naval  Medical  Center  San  Diego  implemented  a  state-of-the-art 
phannacy  automation  system  in  an  effort  to  reduce  cost  and  improve  efficiency.  The 
objective  of  this  study  is  to  quantify  the  increase  in  efficiency  after  installation  through  a 
focus  on  observed  post-automation  prescription  fill  times  during  calendar  year  2014 
(CY2014)  and  a  simulated  pre-automation  process.  With  a  response  of  average  daily 
prescription  fill  time,  automatic  prescription  fills  in  CY2014  are  quicker  than  manual 
prescription  fills  in  CY2014  by  6.97  ±0.97  (standard  error)  minutes,  and  post-automation 
prescription  fills  are  quicker  than  pre-automation  prescription  fills  by  4.4  ±0.34  minutes. 
The  difference  between  pre-automation  and  post-automation  prescription  fills  is  used  as 
the  response  in  a  linear  regression  to  detennine  which  factors  most  contribute  to  the 
decrease  in  prescription  fill  time.  The  proportion  of  prescriptions  automated  is  influential: 
if  this  proportion  is  held  constant  at  0.37,  the  workload  for  each  phannacy  technician  can 
be  reduced  by  an  estimated  2.34  ±0.03  (standard  deviation)  hours  per  day.  A  cost  analysis 
of  the  phannacy  automation  system  is  conducted,  and  it  is  estimated  that  a  lower  bound 
on  the  annual  cost  savings  after  implementation  is  over  $300,000. 
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EXECUTIVE  SUMMARY 


In  today’s  healthcare  environment,  healthcare  facilities  continue  to  seek  ways  to 
reduce  cost  and  improve  efficiency;  the  pharmacy  is  one  of  many  avenues  to  accomplish 
this  task.  In  2012,  Naval  Medical  Center  San  Diego  (NMCSD)  installed  a  state-of-the-art 
pharmacy  automaton  system  in  an  attempt  to  increase  efficiency  within  their  pharmacy. 

Prior  to  2012,  pharmacy  technicians  manually  filled  all  prescription  orders  within 
the  NMCSD  pharmacy.  The  new  phannacy  automation  system  is  capable  of 
autonomously  completing  all  the  steps  in  the  prescription  fill  process  prior  to  pharmacist 
verification.  While  the  automation  system  at  NMCSD  does  not  completely  eliminate  the 
need  for  manual  prescription  fills,  it  has  the  potential  to  greatly  reduce  the  workload 
placed  on  pharmacy  technicians.  The  objective  of  this  study  is  to  quantify  the  increases  in 
efficiency  experienced  after  the  installation  of  pharmacy  automation  at  NMCSD. 

Of  interest  is  the  total  prescription  fill  time  for  each  medication,  which  is  the  time 
elapsed  between  when  a  customer  orders  the  prescription  at  the  intake  window  and  the 
time  pharmacist  verification  is  completed.  Analysis  of  prescription  fill  times  is  based  on 
one  data  set  that  follows  each  individual  prescription  in  calendar  year  2014  (CY2014) 
through  the  entire  prescription  fill  process.  Because  data  from  prior  to  the  implementation 
of  automation  is  unavailable,  medication  demand  and  prescription  characteristics  are 
estimated  from  this  post-automation  data  to  simulate  the  pre-automation  process. 

On  average,  prescriptions  filled  automatically  in  CY2014  are  completed 
6.97  ±0.97  (standard  error)  minutes  faster  than  prescriptions  filled  manually  in  CY2014. 
During  periods  of  high  demand,  automatic  fills  were  quicker  than  manual  fills 
84.5  percent  of  the  time,  with  a  95  percent  confidence  interval  of  [80.2,  87.4].  On 
average,  prescriptions  filled  after  the  installation  of  automation  are  4.4  ±0.43  (standard 
error)  minutes  faster  than  prescriptions  filled  prior  to  the  installation  of  automation. 
These  results  are  found  to  be  statistically  significant. 

The  difference  in  daily  average  prescription  fill  times  pre-automation  and  daily 
average  prescription  fill  times  post  automation  is  used  as  the  response  in  a  linear 
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regression  to  determine  which  characteristics  of  the  data  most  heavily  influence  this 
difference;  a  negative  value  of  the  response  represents  a  day  in  which  the  average 
prescription  fill  time  pre-automation  is  quicker  than  post-automation.  Results  indicate  the 
number  of  hours  of  high  demand  experienced  in  the  day  negatively  impacts  the  response, 
while  the  post-automation  proportion  of  medications  filled  through  automation  and  the 
pre-automation  proportion  of  highly  prescribed  medications  positively  impact  the 
response. 

With  all  other  factors  held  constant,  an  automation  workload  proportion  of  0.37  is 
related  to  an  increase  in  the  difference  of  average  prescription  fill  time  of  2.98  ±  1.95 
(standard  error)  minutes;  if  this  proportion  can  be  maintained  over  the  course  of  an  entire 
year,  assuming  an  average  daily  weekday  demand  is  1186  prescriptions,  this  alone  could 
reduce  yearly  workload  by  636  days,  compared  to  pre-automation.  Assuming  there  are 
25  pharmacy  technicians  working  within  the  pharmacy,  this  is  also  associated  with  a 
reduction  in  daily  workload  of  2.34  ±0.03  hours  per  technician.  Figure  1  illustrates  the 
yearly  reduction  in  weekday  workload  and  daily  reduction  in  workload  based  on 
proportion  of  prescriptions  filled  through  automation. 


Figure  1.  Estimated  Yearly  and  Daily  Reduction  (per  Technician)  in  Weekday 
Workload  Based  on  Proportion  of  Prescriptions  Filled  through 
Automation  Demand. 
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It  is  also  important  to  consider  the  implementation  from  a  cost  perspective.  The 
phannacy  automation  system  at  NMSCD  carried  an  individual  implementation  cost  of 
$2.4  million  of  the  total  $49  million  contract.  There  is  a  reduction  in  total  time  spent 
filling  prescriptions  of  14.2  percent  from  pre-automation  to  post-automation.  This 
reduction  in  time  is  associated  with  $330,000  reduction  in  cost,  the  amount  of  money  it 
would  cost  to  pay  for  the  equivalent  amount  of  work  from  enlisted  military  pharmacy 
technicians.  With  these  calculations,  it  would  take  roughly  seven  years  to  recoup  the  cost 
of  implementation  at  NMCSD  based  on  increased  efficiency.  This  cost  analysis  does  not 
account  for  efficiencies  based  on  the  storage  and  supply  of  medications,  increased  patient 
satisfaction,  increased  access  to  care,  increased  accuracy  of  dispensed  medications,  or 
recapture  costs  based  on  a  reduction  of  non-military  prescription  fills. 

After  the  implementation  of  pharmacy  automation  at  NMCSD,  there  is  an 
increase  in  efficiency,  defined  as  the  total  prescription  fill  time.  This  efficiency  can  be 
quantified  as  reduction  in  total  time  to  fill  a  prescription,  FTEs,  or  equivalent  salary  of 
phannacy  technicians.  This  data  is  collected  after  a  little  more  than  one  year  of 
implementation;  with  better  training  and  more  emphasis  placed  on  the  optimal 
automation  workload,  these  savings  in  efficiency  will  continue  to  increase. 
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XXIV 


I.  INTRODUCTION 


In  today’s  healthcare  environment,  healthcare  facilities  continue  to  seek  ways  to 
reduce  cost  and  improve  efficiency;  the  pharmacy  is  one  of  many  avenues  to  accomplish 
this  task.  The  current  pill-counting  machines  in  most  pharmacies  within  Navy  Medicine 
are  obsolete,  and  more  advanced  replacement  technology  is  available.  In  particular, 
facilities  that  service  a  larger  population  of  customers  have  a  greater  opportunity  to 
benefit  from  technological  advances  in  pharmacy  automation,  because  their  pharmacies 
experience  a  higher  demand.  Replacing  legacy  systems  with  updated  technology  could 
have  extreme  impacts  on  efficiency,  accuracy,  patient  satisfaction,  and  access  to  care. 

Prior  to  2012,  pharmacy  technicians  manually  filled  all  prescriptions  at  the  Naval 
Medical  Center  San  Diego  (NMCSD)  pharmacy,  supplemented  by  an  AccuMed  pill¬ 
counting  machine.  This  pharmacy  received  an  upgrade  in  August  2012,  when  the  Bureau 
of  Medicine  and  Surgery  (BUMED)  installed  a  state  of  the  art  phannacy  automation 
system  at  this  location.  The  new  automation  machine  is  capable  of  autonomously 
completing  all  steps  of  the  prescription  fill  process  prior  to  pharmacist  verification 
without  assistance  from  pharmacy  technicians.  Because  every  prescription  filled  using 
this  new  technology  collects  data  through  barcode  scanning,  its  implementation  provides 
a  unique  opportunity  to  objectively  evaluate  its  contributions  to  efficiency. 

A.  OBJECTIVES  AND  METHODOLOGY 

The  objective  of  this  study  is  to  quantify  the  increases  in  efficiency  after  the 
installation  of  pharmacy  automation  at  NMCSD.  In  order  to  evaluate  these  objectives, 
this  study  focuses  on  individual  prescription  orders  filled  at  NMCSD  during  calendar 
year  2014  (CY2104),  from  January  to  December  2014.  Because  this  data  set  only 
contains  infonnation  for  one  year  post-automation,  a  pre-automation  data  set  is  generated 
through  simulation  for  use  as  comparison.  All  statistical  analysis  and  simulation  is 
completed  using  R  (2013). 

A  customer  placing  a  prescription  order  at  the  intake  window  in  the  pharmacy 
signals  the  beginning  of  the  prescription  fill  process;  the  total  prescription  fill  time  is  of 
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interest,  which  is  the  difference  in  time  between  the  placement  of  an  order  and  the 
completion  of  pharmacist  verification.  To  evaluate  the  efficiency  of  the  machine,  the  total 
prescription  fill  times  for  all  observed  manual  prescriptions  fills  in  CY2014  are  compared 
to  the  total  prescription  fill  times  for  all  observed  automatic  prescriptions  fills  in  CY2014. 
Similarly,  the  entire  post-automation  data  set  is  also  compared  to  the  simulated  pre¬ 
automation  process.  The  results  of  these  comparisons  detennine  if  the  differences  are 
statistically  significant. 

B.  SCOPE,  LIMITATIONS,  AND  ASSUMPTIONS 

This  study  focuses  solely  on  data  obtained  from  NMCSD.  Because  this  study 
focuses  on  one  specific  location,  its  results  can  only  be  directly  applied  to  NMSCD; 
however,  medical  facilities  of  similar  size  and  patient  population  should  expect  to  see 
similar  increases  in  efficiency  after  the  installation  of  a  phannacy  automation  system  at 
their  location.  The  techniques  used  to  evaluate  the  efficiency  of  the  automation  system  at 
NMCSD  can  also  be  easily  applied  to  another  location  with  a  similar  automation  system. 

Data  from  prior  to  the  installation  of  the  automation  system  is  unavailable, 
leading  to  the  most  significant  limitation  of  this  study:  if  this  data  was  obtainable,  a 
considerably  better  comparison  of  changes  in  efficiency  could  be  completed.  In  order  to 
address  this  limitation,  manual  fill  times  and  daily  demand  distributions  are  estimated 
from  the  existing  data  and  used  to  construct  a  simulation  of  the  pre-automation  process. 
In  addition,  with  only  one  complete  year  of  post-automation  data,  it  is  impossible  to 
validate  significant  departures  from  the  mean  average  fill  time  observed  during  specific 
months  as  a  recurring  trend.  To  compensate  for  this  volatility,  the  month  of  February  is 
eliminated  from  the  data  set  for  most  of  the  analysis  to  prevent  this  month  with  unusually 
high  prescription  fill  times  from  unduly  influencing  the  results. 

The  most  basic  assumption  of  this  study  is  that  the  automation  machine  is  running 
properly  and  efficiently  for  the  entire  data  collection  period.  In  the  absence  of 
information  on  the  breakdown  and  repair  of  the  machine,  this  assumption  is  necessary  to 
conduct  analysis.  In  addition,  the  use  of  simulation  to  generate  information  for  the  pre¬ 
automation  process  requires  two  major  assumptions.  First,  it  is  essential  to  assume  that 
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there  is  no  significant  variability  in  the  prescription  filling  abilities  of  the  pharmacy 
technicians  performing  manual  prescription  fills  in  CY2014.  Second,  because  the  demand 
in  the  simulation  is  estimated  from  the  demand  observed  in  CY2014,  it  is  assumed  that 
demand  experienced  during  that  year  is  characteristic  of  any  year  in  the  NMCSD 
phannacy.  Additional  assumptions  required  by  this  study  are  addressed  as  necessary. 

C.  COURSE  OF  STUDY 

This  study  consists  of  four  additional  chapters.  Chapter  II,  Background,  explains 
the  history  of  the  implementation  of  the  pharmacy  automation  system  at  NMCSD  and 
presents  a  review  of  existing  literature  on  the  topic  of  phannacy  automation.  The 
objective  of  Chapter  III,  Data,  and  Chapter  IV,  Methodology  and  Analysis,  is  to  quantify 
the  efficiencies  experienced  after  the  implementation  of  automation.  Chapter  V,  Results, 
Conclusions,  and  Recommendations,  highlights  the  results  and  conclusions  of  the  study, 
makes  recommendations  about  automation  within  Navy  Medicine,  and  suggests  topics 
for  future  work. 
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II.  BACKGROUND 


All  pharmacies  face  a  variety  of  concerns,  such  as  quality  assurance,  pharmacy 
workflow,  patient  satisfaction,  and  cost  control.  With  these  challenges  in  mind,  the 
Bureau  of  Medicine  and  Surgery  (BUMED)  looked  toward  pharmacy  automation  to 
upgrade  their  prescription  fulfillment  process  at  several  high-volume  sites  throughout  the 
enterprise.  Specifically  at  NMCSD,  this  upgrade  converted  the  phannacy  from  a  manual 
system  to  an  automated  system,  supplemented  by  pharmacy  technicians. 

A.  MANUAL  PRESCRIPTION  FILLS 

Prior  to  2012,  pharmacy  technicians  manually  filled  all  prescription  orders  within 
the  NMCSD  pharmacy.  To  fill  a  prescription  manually,  pharmacy  technicians  received 
the  order,  obtained  the  specific  medication  from  its  place  of  storage,  and  then  physically 
counted  each  pill;  thirty-six  AccuMed  automated  pill  counting  machines  supplemented 
the  phannacy  technicians  in  NMCSD,  which  simply  counted  a  specific  number  of  pills, 
loaded  by  the  user  (St.  Onge  Company,  2011).  Once  counting  was  completed,  the 
technician  bottled  the  medication,  capped  the  bottle,  and  then  printed  and  affixed  the 
label.  At  this  stage,  all  bottled  prescriptions  were  physically  carried  to  the  pharmacist  for 
verification,  who  certified  the  medication  filled  fit  the  prescription  entered  for  each 
customer.  Manual  completion  required  the  pharmacy  technicians  and  pharmacists  to 
physically  complete  each  step  in  the  process,  as  well  as  retrieve  medications  from  the 
storage  shelves  and  walk  the  medication  between  the  different  stages. 

The  customer  experienced  a  “bank-teller”  queuing  system  within  the  NMCSD 
pharmacy  while  getting  a  prescription  filled.  Specifically,  when  a  customer  first  arrived 
in  the  phannacy,  he  entered  a  queue  to  be  received  at  the  window.  Once  called  to  the 
window,  he  placed  his  order,  and  then  waited  at  the  window  while  the  order  was  filled. 
When  the  order  was  completed,  the  customer’s  prescription  was  reviewed  with  him,  he 
accepted  the  order,  and  then  he  departed  the  phannacy  (B.  Detlef,  BUMED,  personal 
communication,  July  30,  2015).  Figure  1  illustrates  both  the  customer  queuing  system 
and  the  manual  fill  process. 
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Figure  1.  Bank-Teller  Queuing  System  with  Manual  Fill  Process.  Once  a 

customer  places  an  order,  he  waits  at  the  dispensing  window  while 
his  prescription  is  filled  manually. 


B.  AUTOMATION  IMPLEMENTATION 

In  August  2012,  the  United  States  Navy  awarded  a  $49  million  contract  to 
Innovation,  makers  of  the  PharmASSIST  pharmacy  automation  technology,  to  supply 
pharmacy  automation  to  its  high-volume  sites  (Innovation,  2012).  This  included  the 
installation  of  pharmacy  automation  in  NMCSD  in  2013;  this  site  currently  has  the  most 
technologically  advanced  and  fully  functional  system  in  Navy  Medicine.  This  automation 
system  is  capable  of  handling  and  storing  360  unique  medications,  and  consists  of  a 
conveyer  system  that  is  activated  once  a  customer  places  an  order  (B.  Detlef,  BUMED, 
personal  communication,  April  8,  2015).  The  conveyor  system  is  not  used  solely  for 
prescription  fills  that  have  been  automatically  counted,  capped,  and  labeled;  pharmacy 
technicians  also  use  the  conveyer  system  to  quickly  route  manual  prescription  fills. 
Figure  2  is  a  photograph  of  the  actual  conveyer  system  installed  at  NMCSD. 
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Figure  2.  Conveyer  Belt  Installed  at  Naval  Medical  Center  San  Diego.  Once 
the  customer  places  an  order,  the  conveyor  system  directs  each 
prescription  through  the  different  steps  of  the  fill  process.  Manual  fill 
stations  are  located  to  the  right  side  of  the  conveyer  belt;  as 
pharmacy  technicians  complete  a  manual  fill,  they  place  completed 
orders  onto  the  conveyer  belt  for  routing  to  pharmacist  verification. 


Once  the  customer  order  is  loaded,  the  automation  system  dispenses  a  vial,  affixes 
the  label,  and  delivers  the  empty  bottle  to  the  Robotic  Delivery  System  (RDS).  The  RDS 
inserts  the  labeled  vial  into  the  inventory  container  containing  the  appropriate  medication, 
dispenses  the  pills,  caps  the  vial,  releases  the  vial,  and  routes  the  medication  to  the 
pharmacist  for  verification.  Figure  3  shows  the  RDS  inserting  the  labeled  bottle  into  an 
inventory  container.  This  automatic  procedure  is  used  to  supplement  the  manual  fill 
process  described  in  Section  A. 
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Figure  3.  Robotic  Delivery  System  Installed  at  Naval  Medical  Center  San 

Diego.  Each  container  houses  an  inventory  of  medication.  The  RDS 
selects  a  vial,  inserts  it  into  the  container  to  receive  dispensed 
medication,  caps  the  vial,  and  affixes  the  label.  This  completed 
prescription  is  then  placed  on  the  conveyer  belt  for  routing  to 
pharmacist  verification. 


Along  with  the  implementation  of  the  pharmacy  automation  system,  NMCSD 
underwent  changes  to  their  customer  queuing  system;  instead  of  a  bank-teller  system, 
customers  experience  an  “in-and-out”  system.  When  a  customer  first  arrives  in  the 
pharmacy,  he  takes  a  ticket  number  from  an  automated  kiosk,  and  waits  for  that  ticket 
number  to  be  called.  Once  the  number  is  called,  the  customer  is  received  at  the  window, 
where  he  places  his  order.  This  signals  the  beginning  of  the  prescription  fill  process.  The 
customer  is  dismissed  from  the  window,  where  he  waits  for  his  ticket  number  to  be  called 
for  a  second  time.  After  his  ticket  number  is  called  again,  the  customer  enters  a  queue  to 
wait  for  order  pickup.  At  his  turn,  the  customer  is  received  at  the  dispensing  window,  the 
customer’s  prescription  is  reviewed  with  him,  he  accepts  the  order,  and  then  he  departs 
the  pharmacy  (B.  Detlef,  BUMED,  personal  communication,  July  30,  2015).  Figure  4 
illustrates  the  customer  queuing  process  combined  with  the  manual  and  automatic  fill 
process.  By  dismissing  a  customer  from  the  window,  this  queuing  system  allows  for 
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greater  window  utilization  than  the  bank-teller  model;  the  customer  is  not  occupying  a 
window  while  prescription  operations  are  occurring. 


Figure  4.  In-and-Out  Queuing  System  with  Manual  and  Automatic  Fill 

Process.  Using  the  in-and-out  model,  the  queuing  process  for  the 
customer  is  different,  and  the  prescription  fill  process  is  more 
streamlined.  The  automation  is  capable  of  performing  all  steps, 
including  transportation,  up  to  pharmacist  verification  via  the 
conveyer  belt.  Manual  fills  are  located  along  the  conveyor  system 
for  increased  efficiency. 


While  the  automation  system  at  NMCSD  does  not  completely  eliminate  the  need 
for  manual  prescription  fills,  it  has  the  potential  to  greatly  reduce  the  workload  placed  on 
pharmacy  technicians.  This  reduction  in  workload  occurs  not  only  in  the  physical  number 
of  orders  filled  manually,  but  also  decreases  the  need  for  pharmacy  technicians  to 
constantly  retrieve  and  stock  medication;  most  of  the  high-demand  medications  are  stored 
within  the  RDS  system.  In  addition,  each  step  in  the  automatic  process  is  triggered  by 
barcode  scanning,  which,  in  theory,  should  lead  to  more  accurate  prescription  fills. 
Through  barcode  scanning,  the  PharmASSIST  software  ensures  that  every  step  in  the 
process  is  automatically  double  checked  for  accuracy  prior  to  the  pharmacist  verification. 
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c. 


LITERATURE  REVIEW 


There  exists  extensive  literature  surrounding  the  implications  of  changing 
phannacy  operations  within  a  hospital;  much  of  this  literature  specifically  identifies  the 
outcomes  related  to  the  implementation  of  different  automation  systems.  Due  to  an 
environment  of  rapidly  changing  technology,  the  ability  to  understand  the  impact  of  such 
a  change  is  of  significant  interest  to  any  healthcare  organization.  A  review  of  existing 
literature  concerns  the  topic  of  efficiency.  This  review  is  limited  to  studies  conducted  in 
hospital  phannacies  as  opposed  to  community  pharmacies,  which  are  of  similar  size  and 
scope  to  NMCSD,  but  are  varied  in  the  types  of  automation  systems  implemented. 

The  use  of  emerging  technologies  within  pharmacies  is,  in  part,  directly  aimed  at 
increasing  productivity  while  reducing  customer  wait  times.  One  way  to  quantify  the 
productivity  of  the  phannacy  is  to  measure  the  total  output  of  prescription  items  over 
time;  through  statistical  inference,  the  quantity  of  items  produced  prior  to  installation  can 
be  compared  to  after  installation  and  evaluated  for  significance.  Though  many  other 
factors,  such  as  staff  training  and  competency,  can  impact  this  metric,  the  impact  of 
automation  on  the  number  of  items  produced  per  day  varies  widely  from  study  to  study. 
This  increase  can  range  from  as  little  as  nineteen  percent  to  as  much  as  43  percent;  see 
Fitzpatrick,  Cooke,  Southall,  Kauldher,  and  Waters  (2005),  Angelo,  Christensen,  and 
Ferrerri  (2005),  and  James  et  al.  (2013).  Many  of  these  numbers  have  significant  standard 
deviations,  and  items  produced  per  hour  can  vary  by  as  much  as  68  percent  of  the  mean 
(Angelo,  Christensen,  &  Ferreri,  2005).  In  addition,  staffing  practices  must  be  adjusted  to 
maximize  this  effect  on  productivity.  While  automation  can  reduce  the  average  time 
spent  filling  prescriptions  by  seventeen  percent  after  the  introduction  of  automation,  this 
reduction  is  associated  with  eighteen  percent  increase  in  time  spent  doing  non-productive 
activities  if  staffing  levels  remain  unchanged  (Lin,  Huang,  Punches,  &  Chen,  2007). 

Because  of  the  cost  associated  with  automation  machines,  many  healthcare 
organizations  turn  to  simulation  as  a  reduced  cost  method  to  influence  decision-making. 
With  proper  modeling  and  input  parameters,  simulations  can  provide  valuable  insights 
into  important  aspects  of  pharmacy  management,  especially  when  used  as  a  queuing 

model.  Customer  satisfaction  surveys  indicate  that  phannacy  customers  value  a  wait  time 
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of  less  than  30  minutes  as  the  third  most  important  factor  in  a  pharmacy  experience,  after 
only  prescription  accuracy  and  affordability  of  medicine  (Vincent  &  Lim,  1997).  This 
goal  of  a  30-minute  customer  wait  time  is  used  as  target  metric  in  Tan,  Chua,  Yong,  and 
Wu’s  simulation  of  the  impacts  of  automation  on  a  hospital  pharmacy  in  Singapore 
(2009).  The  results  of  this  simulation  indicate  that  the  implementation  of  automation 
alone  is  not  substantial  enough  to  reduce  the  95th  percentile  customer  wait  time  to  below 
30  minutes;  in  addition  to  automation,  pharmacies  must  change  their  staffing  procedures 
and  workflow  management  to  achieve  this  goal  (2009). 

Based  on  the  unavailability  of  data  collected  prior  to  the  installation  of  the 
automation  machine  at  NMCSD,  this  study  takes  a  hybrid  approach,  incorporating  both 
empirical  evaluation  and  simulation.  The  post-automation  data  indicates  whether  a 
prescription  is  filled  manually  or  automatically;  therefore,  inferences  are  made  about 
manual  fill  times,  in  general.  The  data  also  provide  demand  signals  that  assist  in 
determining  customer  requests  on  an  hourly,  daily,  and  weekly  basis.  This  infonnation  is 
used  simulate  the  pre-automation  environment,  which  is  the  basis  for  statistical  inference 
in  Chapter  IV. 

D.  SUMMARY 

The  efficiency  of  medication  dispensed  in  a  hospital  pharmacy  will  always  be  of 
the  utmost  importance  to  any  healthcare  organization.  Navy  Medicine  has  attempted  to 
address  these  issues  through  the  implementation  of  pharmacy  automation  of  varying 
degrees  at  its  high  volume  sites.  At  NMCSD,  in  particular,  the  most  advanced  and  fully 
functional  automation  machine  has  been  in  use  for  nearly  three  years;  this  type  of 
automation  has  the  potential  to  greatly  reduce  workload  on  pharmacy  technicians  and 
phannacists,  which  in  turn,  will  increase  efficiency  and  patient  satisfaction.  These 
benefits  will  be  addressed  in  the  following  chapters. 
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III.  DATA 


Analysis  of  prescription  fill  times  is  based  on  one  data  set  generated  by 
Innovation,  the  system  that  runs  the  automation  machine  procedures  to  fill,  verify,  and 
stage  prescriptions  at  NMCSD.  Improvement  Path  Systems,  an  organization  contracted  to 
perform  healthcare  analytics,  provided  this  data  via  the  BUMED.  This  data  set  follows 
each  individual  prescription  in  CY2014  through  the  entire  fill  process;  the  time  lapse 
between  automation  implementation  in  2012  and  data  collection  in  2014  reduces  the 
possibility  of  an  adjustment  period  influencing  the  results.  Specifically,  this  accounts  for 
technician  training  and  modifications  to  standard  operating  procedures.  Medication 
demand  and  prescription  characteristics  are  estimated  from  this  post-automation  data  to 
simulate  the  pre-automation  process,  because  data  from  prior  to  the  implementation  of  an 
automation  system  is  unavailable.  Using  both  the  simulated  pre-automation  and  observed 
post-automation  data,  it  is  then  possible  to  conduct  the  statistical  analysis  completed  in 
Chapters  IV  and  V. 

Each  observation,  403,900  in  total,  represents  an  individual  prescription  request  at 
the  pharmacy  during  CY2014,  and  includes  a  unique  prescription  identification  number, 
dispensed  medication  name,  and  fill  type  (manual  or  automatic).  In  addition  to  these 
descriptive  fields,  the  data  set  contains  fifteen  tracked  processes  for  each  prescription. 
Each  process  is  represented  by  the  date  and  time  in  which  the  process  occurred,  accurate 
to  the  second.  Table  1  details  the  eight  processes  relevant  to  this  study;  other  processes  of 
interest,  but  not  used  to  this  study,  are  found  in  Appendix  A. 
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Table  1.  Processes  Tracked  in  Prescription  Data.  The  prescription  data 

contains  the  date  and  time,  accurate  to  the  second,  of  eight  relevant 
processes  during  the  entire  prescription  order  procedure. 


Process 

Description  (date/time) 

Scheduled 

Entered  into  the  system 

Counted 

Count  of  quantity  of  medication  is  complete 

Added  to  Tote 

Counting,  bottling,  capping,  and  labeling  is  complete 

Verified 

Pharmacist  verification  for  errors  is  complete 

Added  to  Bag 

Combined  with  remaining  customer  order 

Added  to  Location 

Complete  order  is  placed  in  assigned  cubby  location 

Ready  for  Pickup 

Order  is  complete  and  ready  for  customer  pick  up 

Pickup 

Customer  picks  up  completed  order. 

Figure  5  illustrates  the  flow  of  the  prescription  order  process  within  the  pharmacy. 
For  this  analysis,  total  time  to  fill  a  single  prescription  is  of  interest.  Total  time  is  defined 
as  the  difference  in  time  between  scheduling  and  verification;  all  processes  after  that  are 
possibly  dependent  on  the  other  items  contained  in  a  customer  order,  or  the  actions  of  the 
customers  themselves.  The  difference  in  scheduled  time  and  verified  time  is  found  for 
each  observation,  and  this  time  difference  is  used  as  the  response  during  analysis 


Figure  5.  Prescription  Fill  Flow  Chart.  The  general  prescription  fill  process  is 
outlined,  including  indication  of  which  steps  in  are  recorded  in  the 
data  set. 
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Of  the  403,000  observations  identified  in  the  post-automation  prescription  data, 
32,841  items  are  eliminated  based  on  issues  with  observed  scheduled,  verified,  and 
pickup  times.  Observations  without  these  times  recorded  are  excluded.  In  addition,  all 
observations  with  another  process  recorded  at  a  point  in  time  prior  to  scheduling  are 
removed;  scheduling  is  the  very  first  step  in  the  prescription  fill  process,  and  observations 
contrary  to  this  are  treated  as  errors.  These  times  are  important  because  their  difference  is 
used  as  the  measure  of  total  prescription  fill  time.  Another  109  observations  are 
eliminated  because  the  type  of  fill,  either  manual  or  automatic,  is  not  specified.  Lastly, 
7,017  observations  have  a  total  prescription  fill  time  of  either  less  than  two  minutes  or 
greater  than  one  day.  Because  the  median  prescription  fill  time  is  16.3  minutes,  including 
these  observations,  observations  greater  than  one  day  are  considered  extreme  outliers  and 
excluded  from  analysis.  Similarly,  based  on  the  number  of  steps  in  the  prescription  fill 
process,  any  observation  of  less  than  two  minutes  is  uncharacteristically  quick.  The  result 
is  a  final  data  set  consisting  of  364,529  observations,  as  illustrated  in  Figure  6. 


Figure  6.  Reduction  of 403,900  Initial  Observations  to  Final  Data  Set  of 

364,529.  An  analysis  of  available  prescription  data  results  in  a  final 
sample  size  of  364,529  observations  for  CY2014  at  NMCSD. 
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In  the  final  data  set,  the  automation  machine  completes  37.3  percent  of  all 
prescription  fills;  the  rest  are  filled  manually.  Summary  statistics  for  the  final  data  set, 
distinguishing  prescriptions  filled  automatically  from  manually,  are  contained  in  Table  2. 


Table  2.  Summary  Statistics  of  Prescription  Fill  Times  (Minutes).  In  the 

final  data  set,  37.3  percent  of  all  prescription  fills  for  CY2014  are 
filled  through  automation. 


Type 

Number 
of  Fills 

Minimum 

1st 

Quartile 

Median 

Mean 

3rd 

Quartile 

Maximum 

Total 

364,529 

8.95 

15.98 

26.09 

28.10 

1440.00 

Auto 

136,068 

2.00 

7.07 

12.90 

21.74 

23.48 

1435.00 

Manual 

228,461 

10.38 

17.95 

28.68 

30.75 

1440.00 

A.  POST-AUTOMATION  PRESCRIPTION  FILL 

The  final  data  set  contains  the  entirety  of  information  used  to  analyze  post¬ 
automation  prescription  fills.  It  is  of  interest  to  examine  trends  in  demand  and 
prescription  fill  times  based  on  different  time  periods;  assuming  these  tendencies  are 
consistent  with  the  typical  phannacy  environment  at  NMCSD,  they  are  used  as  the  basis 
to  simulate  the  pre-automation  process.  Data  is  binned  by  day,  day  of  the  week,  and  hour 
of  the  day  to  determine  significant  time  periods  in  order  to  better  understand  the 
workload  and  demand  signals  within  the  pharmacy. 

1.  Daily  Trend  Analysis 

First,  the  number  of  prescriptions  observed  is  totaled  by  day  of  the  year.  This 
procedure  results  in  365  observations,  and  serves  as  the  basis  for  the  first  stage  of 
analysis,  see  Figure  7.  Upon  inspection,  a  weekly  trend  is  apparent;  there  are  roughly  four 
local  minima  and  maxima  during  each  month,  corresponding  to  the  number  of  weeks  in  a 
month.  In  addition,  the  number  of  prescriptions  appears  to  increase  in  the  month  of 
February,  while  the  rest  of  the  year  remains  fairly  consistent. 
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Figure  7.  Total  Number  of  Prescriptions.  This  includes  prescriptions  filled 

both  manually  and  automatically.  The  maximum  number  of  daily 
prescriptions  is  1,91 1  observations,  occurring  on  February  12,  2014. 

The  minimum  number  of  prescriptions  is  246,  occurring  on 
November  27,  2014.  The  sharp  increases  and  decreases  in  the 
number  observations  during  each  month  indicate  a  weekly  trend. 

Similarly,  prescription  fill  time  is  averaged  by  day  of  the  year,  resulting  in 
365  observations;  see  Figure  8.  Average  daily  prescription  fill  times  experience  an 
increase  during  the  month  of  February,  while  they  are  at  their  lowest  during  the  month  of 
August.  This  spike  in  prescription  fill  time  during  the  month  of  February  corresponds 
with  an  increase  in  customer  orders  during  that  month.  With  only  one  year  of 
observations,  this  cannot  be  validated  as  a  recurring  seasonal  trend.  This  departure  from 
the  mean  proves  to  be  problematic  throughout  this  study;  as  a  result,  the  month  of 
February  is  eliminated  during  most  analysis  conducted  in  Chapters  IV  and  V. 
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Figure  8.  Average  Prescription  Fill  Times.  This  includes  prescriptions  filled 

both  manually  and  automatically.  The  maximum  average  fill  time  of 
57.9  minutes  occurred  on  February  5,  2014,  while  the  minimum  of 
9.04  minutes  occurred  on  August  10,  2014. 

2.  Day  of  Week  Trend  Analysis 

Next,  the  number  of  prescriptions  filled  and  their  fill  times  are  analyzed  based  on 
day  of  the  week.  It  is  noted  previously  that  there  is  a  significant  weekly  trend  in  the 
number  of  prescriptions  filled  within  a  day.  The  pharmacy  at  NMCSD  is  open  seven  days 
a  week;  however  the  regular  workweek  for  the  base  is  Monday  through  Friday. 
Therefore,  it  is  expected  that  there  are  significantly  fewer  customer  orders  on  weekends 
than  on  weekdays,  because  patients  tend  to  make  appointments  during  the  business  week. 
In  order  to  validate  this  expectation,  observations  are  binned  by  day  of  the  week,  and  then 
averaged  within  that  bin.  Demand  decreases  on  the  weekends,  which  is  also  associated 
with  a  lower  average  prescription  fill  time.  Of  more  interest,  the  average  prescription  fill 
time  on  Mondays  is  higher  than  other  weekdays  without  a  corresponding  increase  in 
demand.  Figure  9  illustrates  the  effect  that  day  of  the  week  has  on  average  number  of 
prescriptions  filled  and  average  prescription  fill  time. 
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Figure  9.  Average  Prescription  Fill  Time  and  Average  Number  of  Customer 
Prescription  Orders,  Binned  by  Day  of  Week.  The  black  bars 
represent  standard  error  of  the  mean.  Demand  decreases  on 
weekends,  which  is  also  associated  with  a  lower  average  prescription 
fill  time.  On  weekdays,  demand  for  prescriptions  remains  fairly 
constant.  Mondays  have  an  increased  average  prescription  fill  time 
without  a  corresponding  increase  in  demand. 


3.  Hour  of  Day  Trend  Analysis 

A  normal  workday  for  the  outpatient  pharmacy  at  NMCSD  begins  at  0800  and 
ends  at  2100,  Monday  through  Friday.  On  Saturday  and  Sunday,  business  hours  are  from 
0800  to  1800.  At  all  other  times,  the  outpatient  pharmacy  is  officially  only  open  for 
prescription  fills  associated  with  Emergency  Department  visits.  Because  of  this  schedule, 
it  is  expected  that  customer  demand  will  be  higher  during  working  hours  and  lower 
during  non- working  hours.  In  order  to  validate  this  expectation,  observations  are  binned 
by  hour  of  the  day,  and  then  averaged  within  that  bin.  Prescription  demand  experiences  a 
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sharp  increase  at  0800,  and  then  tapers  off  after  1700.  Prescription  fill  times  experience 
delays  around  0700  and  1800.  Figure  10  illustrates  the  relationship  between  hour  of  the 
day,  prescription  demand,  and  prescription  fill  times. 


Figure  10.  Average  Prescription  Fill  Time  and  Average  Number  of  Customer 
Prescription  Orders,  Binned  by  Flour  of  the  Day.  The  black  bars 
represent  standard  error  of  the  mean.  Weekends  and  weekends  are 
separated  because  there  are  different  working  hours,  depending  on 
the  day.  Prescription  demand  experiences  a  sharp  increase  at  0800, 
and  then  tapers  off  after  1700.  Prescription  fill  times  experience 
delays  around  0700  and  1800. 


Based  on  the  observed  increase  in  prescription  fill  times  without  an  increase  in 
demand  on  Mondays,  this  day  is  evaluated  individually.  Figure  1 1  illustrates  the  average 
prescription  fill  time,  per  hour,  during  working  hours  on  Monday;  there  is  an  extreme 
increase  in  the  fill  time  from  1700  to  1900.  This  hourly  increase  directly  contributes  to 
the  overall  increase  in  average  daily  prescription  fill  times  on  Mondays,  compared  to  the 
rest  of  the  week. 
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Figure  1 1 .  Average  Hourly  Prescription  Fill  Time  During  Working  Hours, 

Monday.  The  black  bars  represent  standard  error  of  the  mean.  There 
is  an  extreme  increase  in  the  average  hourly  prescription  fill  time 
from  1700  to  1800,  where  the  fill  times  jumps  to  a  maximum  of 
147  minutes. 


B.  PRE-AUTOMATION  PRESCRIPTION  FILL 

A  significant  limitation  of  this  study  is  the  unavailability  of  data  collected  pre¬ 
automation.  To  evaluate  the  effectiveness  of  the  automation  system,  a  pre-automation 
data  set  is  constructed  using  simulation  to  model  the  process  prior  to  automation 
implementation.  Two  major  assumptions  are  made  in  order  to  construct  this  simulation: 
first,  the  CY2014  medication  demand  is  representative  of  typical  yearly  demand,  and 
second,  the  manual  fill  times  of  the  pharmacy  technicians  in  CY2014  is  characteristic  of 
typical  manual  fill  times  per  prescription. 

1.  Design 

Using  the  name  of  the  dispensed  medication  as  an  identifier  in  the  CY2014  data 
set,  it  is  observed  that  there  are  2,340  unique  medications — the  same  type  of  medication 
in  a  different  dosage  is  counted  as  a  unique  medication.  Of  those  medications,  the  ten 
most  prescribed  medications  account  for  21.3  percent  of  the  total  prescription  in  CY2014; 
a  distribution  of  the  25  most  prescribed  medications  can  be  found  in  Appendix  B. 
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Prescription  fill  time  is  dependent  on  the  type  of  medication  ordered.  Figure  12  and 
Figure  13  illustrate  the  difference  in  manual  prescription  fill  time  and  demand  for  four  of 
the  most  frequently  filled  medications. 


Figure  12.  Histogram  of  Daily  Average  Manual  Fill  Time  (Minutes)  for  Four 
Frequently  Prescribed  Medications.  The  manual  fill  time  varies 
based  on  medication. 
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Figure  13.  Prescription  Demand  for  Three  Frequently  Prescribed  Medications. 

Demand  for  different  medications  varies  throughout  the  year,  based 
on  the  unique  medication.  For  example,  the  demand  for 
Hydrocodone/Acetaminophen  326MG  tends  to  be  larger  than  the 
demand  for  Acetaminophen  325MG  during  the  middle  portion  of  the 
year. 


Monthly  and  weekly  trends  are  used  as  the  basis  for  simulation;  it  is  not  essential 
to  detail  hourly  trends  at  this  level.  Because  it  is  assumed  that  the  demand  and  manual 
prescription  fill  times  are  consistent  with  CY2014  data,  it  is  only  necessary  to  simulate 
the  extra  demand  placed  on  the  pharmacy  technicians  in  the  absence  of  automation.  For 
this  simulation,  this  means  that  the  actual  fill  times  observed  in  CY2014  for  manual 
transactions  each  day  of  the  year  are  left  unchanged;  only  the  additional  manual  workload 
not  supplemented  by  the  automation  machine  is  simulated.  The  variables  of  interest  for 
the  simulation  include  total  prescription  demand,  medication  demand,  and  prescription 
fill  times,  as  defined  in  Table  3. 
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Table  3.  Indices,  Derived  Data,  and  Calculated  Variables  for  Simulated 
Pre-Automation  Process.  The  additional  manual  workload  not 
supplemented  by  the  automation  machine  is  simulated  to  create  a 
pre-automation  data  set. 


Notation 

Description 

Indices 

i 

Day  of  the  Year,  i  g{1,...,365} 

i 

Day  of  the  Week,  j  e{l,...,7} 
where  /  =  1  is  Sunday 

k 

Month,  k  e  {1, ...,  12} 

m 

Medication,  m  e  {1,...,2340} 

Derived 

Data 

total  _  fills  f 

Total  number  of  prescriptions 
filled  on  each  day  of  the  year,  i. 

manual  _  fills  t 

Total  number  of  prescriptions 
filled  manually  on  each  day  of  the 
year,  i. 

medication  demand  k 

Vector  of  medications  demanded 
on  each  day  of  the  week,y,  for 
each  month  of  the  year,  k. 

manual  fill  time  . . 

—  —  m,j,k 

Vector  of  manual  fill  times 
associated  with  medication  m  for 
each  day  of  the  week,  j,  for  each 
month  of  the  year,  k. 

Calculated 

Variables 

sdi,k 

Standard  deviation  of  daily 
demand  on  each  day  of  the  week,y, 
and  month,  k. 

upper  _boundi 

Upper  bound  for  simulated  total 
prescription  demand  on  each  day 
of  the  year,  i. 

lower  _boundi 

Lower  bound  for  simulated  total 
prescription  demand  on  each  day 
of  the  year,  i. 

simulated  -total; 

Total  simulated  prescription 
demand  on  each  day  of  the  year,  i. 

excess  -manual; 

Excess  manual  workload  on  each 
day  of  the  year,  i. 
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First,  the  medication  demand  at  NMCSD  phannacy  in  CY2014  is  aggregated 
by  day  of  the  week,  j,  and  month,  k,  resulting  in  84  vectors  of  varying  length, 
medication _ demand .  k .  This  vector  contains  the  names  of  all  medications  fdled  on  day 

of  the  week,  j,  in  month,  k;  individual  medications  appear  in  this  vector  with  the  same 
frequency  as  observed  in  CY2014.  Similarly,  all  manual  fill  times  associated  with  the 
demanded  medications  are  aggregated  by  medication,  m,  day  of  the  week,  and  month  of 
the  year,  manual _fill_timem  r  This  results  in  196,560  vectors  of  varying  length,  one  for 

every  medication,  m,  on  every  day  of  the  week,  j,  in  each  month,  k.  Then,  total  number  of 
prescription  fills  and  manual  prescription  fills  completed  on  each  day  of  the  year,  i,  are 
calculated.  This  results  in  365  observations  of  two  variables,  total  _  fills  t  and 
manual  _  fills respectively;  on  average,  the  automation  machine  accounts  for 
37.3  percent  of  prescription  fills  per  day.  The  total  number  of  prescriptions  ordered  on 
a  specific  day  of  the  week  within  each  month  is  modeled  as  uniformly  distributed.  The 
lower  and  upper  bounds  of  the  uniform  distribution  are  calculated  the  observed  total  fill 
for  that  day,  total  _  fills i ,  and  the  standard  deviation  of  the  daily  demand  corresponding  to 
the  day  of  the  week  and  month,  sd.  k ;  this  is  shown  in  Equations  (1)  and  (2). 

upper  bound ;  =  total  _  fills :  +  sd.  k  (1) 

lower  bound ;  =  total  fills t  -  sdf  k  (2) 

i  g  {1,...,365},  y  g  {1, 7},  k  g  {1,...12} 

For  each  day  of  the  year,  a  random  uniform  number  within  the  calculated  lower 

and  upper  bounds  is  drawn  to  detennine  the  day’s  simulated  total  demand, 

simulated _totalr  Because  the  observed  manual  prescription  fills  from  CY2014  will 
remain  unchanged,  the  observed  manual  fills,  manual  _  fills .  are  removed  from  the 
simulated  total  demand,  leaving  only  the  excess  workload  per  day,  excess  _  manual t ,  as 
shown  in  Equation  (3). 

excess _ manual  =  simulated _totalt-  manual _  fills t,  i  e{l,.„,  365} 
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(3) 


For  each  day  of  the  year,  a  random  medication  sample  of  size  excess  _  manual  t  is 
selected  at  random  from  medication  _  demand .  k ,  based  on  the  appropriate  day  of  the 

week  and  month.  Using  the  medications  in  the  sample,  manual  fill  times  are  chosen  with 
replacement  from  manual  fill  time,,,  k ,  the  actual  manual  fill  times  for  each 

medication  observed  during  that  month  and  day  of  the  week.  If  a  medication  does  not 
have  an  associated  manual  fill  time  for  a  specified  day  of  the  week  within  month  k,  the 
manual  fill  time  is  assigned  as  the  average  manual  fill  time  for  the  specified  medication 
in  month  k.  If  a  medication  does  not  have  an  associated  manual  fill  time  in  all  of  month  k, 
the  manual  fill  time  is  assigned  as  the  average  manual  fill  time  for  the  specified 
medication  over  the  entire  year.  If  a  medication  does  not  have  an  associated  manual  fill 
time  in  the  entire  year,  the  manual  fill  time  is  assigned  as  the  average  manual  fill  time  for 
all  medications;  there  are  only  seventeen  medications  in  CY2014  filled  solely  through 
automation,  accounting  for  0.07  percent  of  total  observations.  This  entire  process  is 
repeated  with  1,000  replications. 

Through  this  technique,  135,972  data  points  are  created  using  simulation, 
representing  the  excess  demand  within  the  pharmacy  not  supplemented  by  automation. 
These  simulated  observations  are  then  combined  with  the  observed  manual  prescription 
fill  times,  resulting  in  a  pre-automation  data  set  consisting  of  364,443  total  data  points. 
Figure  14  illustrates  the  observed  and  simulated  prescription  demand. 
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Figure  14.  Simulated  Excess  Demand  Combined  with  Observed  CY2014 

Prescription  Demand.  Through  simulation,  the  excess  demand  not 
supplemented  by  automation  is  generated  to  produce  a  pre¬ 
automation  data  set. 


2.  Validation 

In  order  to  confirm  the  validity  of  this  simulated  process,  the  automation  demand 
for  CY2014  is  generated  through  a  similar  technique,  using  only  monthly  and  weekly 
trends.  Again,  the  manual  fill  times  observed  in  CY2014  are  left  unchanged;  only  the 
automatically  filled  prescriptions  are  simulated.  By  simulating  the  automated  prescription 
fills  and  combining  these  data  points  with  the  observed  manual  fill  times,  simulated  data 
set  can  be  compared  to  the  observed  CY2014  data.  The  variables  of  interest  for  this 
validation  include  total  prescription  demand,  medication  demand,  and  prescription  fill 
times,  as  defined  in  Table  4. 
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Table  4.  Indices,  Derived  Data,  and  Calculated  Variables  for  Simulated 

Automation  Process.  The  additional  manual  workload  not 
supplemented  by  the  automation  machine  is  simulated  to  create 
automation  data. 


Notation 

Description 

Indices 

i 

Day  of  the  Year,  i  e  {1,...,  365} 

j 

Day  of  the  Week,  j  g{1,...,7} 
where  ;=1  is  Sunday 

k 

Month,  k  e  {1, 12} 

m 

Medication,  m  e  {1,...,2340} 

Derived 

Data 

total  _  fills  j 

Total  number  of  prescriptions 
filled  on  each  day  of  the  year,  i. 

manual  _  fills  t 

Total  number  of  prescriptions 
filled  manually  on  each  day  of  the 
year,  i. 

medication  demand  k 

Vector  of  medications  demanded 
on  each  day  of  the  week,y,  for 
each  month  of  the  year,  k. 

auto  fill  time  .. 

—  —  m,j,k 

Vector  of  automatic  fill  times 
associated  with  medication  m  for 
each  day  of  the  week,  j,  for  each 
month  of  the  year,  k. 

Calculated 

Variables 

sdi,k 

Standard  deviation  of  daily 
demand  on  each  day  of  the  week,y, 
and  month,  k. 

upper  _boundi 

Upper  bound  for  simulated  total 
prescription  demand  on  each  day 
of  the  year,  i. 

lower  _boundj 

Lower  bound  for  simulated  total 
prescription  demand  on  each  day 
of  the  year,  i. 

simulated  _totalf 

Total  simulated  prescription 
demand  on  each  day  of  the  year,  i. 

simulated  _autoi 

Simulated  automatic  workload  on 
each  day  of  the  year,  i. 

The  medication_demand/jt  vector  remains  unchanged  for  validation.  All 

automatic  fill  times  associated  with  the  demanded  medications  are  binned  by  medication, 
day  of  the  week,  and  month  of  the  year,  auto _ fill _ timem  .  k .  This  results  in  196,560 

vectors  of  varying  length,  one  for  every  medication,  m,  on  every  day  of  the  week,  j,  in 
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each  month,  k.  Again,  the  total  number  of  prescriptions  ordered  on  a  specific  day  of  the 
week  within  each  month  is  modeled  as  uniformly  distributed.  The  lower  and  upper 
bounds  of  the  uniform  distribution  are  calculated  the  observed  total  fill  for  that  day, 
total  _  fills .,  and  the  standard  deviation  of  the  daily  demand  corresponding  to  the  day  of 
the  week  and  month,  sd .  k ;  this  is  shown  in  Equations  (1)  and  (2). 

For  each  day  of  the  year,  a  random  uniform  number  within  the  calculated  lower  and 
upper  bounds  is  drawn  to  detennine  the  day’s  simulated  total  demand,  simulated _totalr 
Because  the  observed  manual  prescription  fills  from  CY2014  will  remain  unchanged,  the 
observed  manual  fills  are  removed  from  the  simulated  total  demand,  leaving  only  the 
automation  workload,  simulated _autoj,  as  shown  in  Equation  (4). 

simulated _autoi  =  simulated _totalt  -  manual _  fills p  i  e  {1,...,  365}  (4) 

For  each  day  of  the  year,  a  random  medication  sample  of  size  simulated _autoi  is 
selected  at  random  from  medication  _  demand .  k ,  based  on  the  appropriate  day  of  the 

week  and  month.  Using  the  medications  in  the  sample,  automatic  fill  times  are  chosen 
with  replacement  from  auto  _  fill  _ time m  jk,  the  actual  automatic  fill  times  for  each 

medication  observed  during  that  month  and  day  of  the  week.  If  a  medication  does  not 
have  an  associated  automatic  fill  time  for  a  specified  day  of  the  week  within  month  k,  the 
fill  time  is  assigned  as  the  average  automatic  fill  time  for  the  specified  medication  in 
month  k.  If  a  medication  does  not  have  an  associated  automatic  fill  time  in  all  of  month  k, 
the  fill  time  is  assigned  as  the  average  automatic  fill  time  for  the  specified  medication 
over  the  entire  year.  If  a  medication  does  not  have  an  associated  automatic  fill  time  in  the 
entire  year,  the  fill  time  is  assigned  as  the  average  automatic  fill  time  for  all  medications. 
This  entire  process  is  repeated  with  1,000  replications. 

Through  this  technique,  136,074  data  points  are  constructed  using  simulation, 
representing  the  automation  workload  within  the  pharmacy.  These  simulated  observations 
are  then  combined  with  the  observed  manual  prescription  fill  times,  resulting  in  a  data  set 
consisting  of  364,535  total  data  points.  Figure  15  illustrates  the  observed  daily  demand 
and  the  daily  demand  with  simulated  automation  workload;  Figure  16  illustrates  the 
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observed  average  daily  prescription  fill  time  and  the  average  daily  prescription  fill  time 
with  simulated  automation  workload. 


Jan  Feb  Mar  Apr  May  Jun  Jul  Aug  Sep  Oct  Nov  Dec 

Figure  15.  Observed  and  Simulated  Daily  Demand.  With  the  same  simulation 
techniques  used  to  create  the  pre-automation  manual  workload,  a 
simulated  automation  workload  is  generated. 


Figure  16.  Observed  and  Simulated  Automation  Average  Prescription  Fill 

Time.  With  the  same  simulation  techniques  used  to  create  the  pre¬ 
automation  manual  workload,  a  simulated  automation  workload  is 
generated. 
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This  simulated  automated  workload  provides  a  degree  of  validity  to  the 
techniques  used  to  simulate  the  pre-automation  data.  The  simulated  automatic  data  set 
tends  to  overestimate  both  the  daily  demand  and  the  average  daily  prescription  fill  time; 
this  suggests  that  the  simulated  pre-automation  data  set  will  be  a  upper  bound  of  the 
manual  workload.  The  absolute  mean  difference  between  daily  observed  demand  and 
daily  simulated  demand  is  72.9  ±51.2  (standard  deviation)  items;  70  percent  of  the 
differences  fall  within  100  items.  February  is  a  month  with  uncharacteristically  large 
differences,  and  if  excluded  from  the  data  set,  the  mean  difference  between  daily 
observed  demand  and  daily  simulated  demand  is  67.4  +  40.6  items.  Figure  17  illustrates 
the  difference  in  the  demand  for  the  observed  and  simulated  data  sets  throughout  an 
entire  year.  The  mean  difference  between  average  daily  observed  prescription  fill  time 
and  average  daily  simulated  prescription  fill  time  is  6.3  ±4.8  minutes;  83  percent  of 
observations  fall  within  ten  minutes.  With  February  excluded,  the  mean  difference 
between  average  daily  observed  prescription  fill  time  and  average  daily  simulated 
prescription  fill  time  is  5.4  ±2.9  minutes.  Figure  18  illustrates  the  difference  in 
prescription  fill  time  for  the  observed  and  simulated  data  set  throughout  an  entire  year. 
Evaluation  of  the  difference  in  the  observed  and  simulated  data  set  aggregated  by  month 
and  day  of  week,  is  conducted;  other  than  the  month  of  February,  there  does  not  appear  to 
be  any  patterns  in  the  average  differences  based  on  month  or  day  of  week.  These  plots 
can  be  found  in  Appendix  C. 
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Figure  17.  Difference  in  Daily  Observed  Demand  and  Daily  Simulated 
Demand.  After  simulating  the  automated  workload,  the  mean 
difference  between  the  daily  observed  demand  and  the  daily 
simulated  demand  72.9  observations  with  a  standard  deviation  of 
51.2  observations.  The  month  of  February  is  extremely  volatile; 
without  this  month,  the  mean  difference  is  67.4  observations  with  a 
standard  deviation  of  40.6  minutes. 
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Figure  18.  Difference  in  Average  Daily  Observed  Prescription  Fill  Time  and 
Average  Daily  Simulated  Prescription  Fill  Time.  After  simulating 
the  automation  workload,  the  mean  difference  between  the  observed 
average  daily  prescription  fill  time  and  the  simulated  daily 
prescription  fill  time  is  6.3  minutes  with  a  standard  deviation  of  4.8 
minutes.  The  simulated  data  set  is  more  volatile  during  the  month  of 
February;  without  this  month,  the  mean  difference  is  5.4  minutes 
with  a  standard  deviation  of  2.9  minutes. 


To  determine  if  there  are  any  systematic  trends  over  time,  two  statistical  tests  are 
performed  on  the  difference  in  average  daily  observed  prescription  fill  time  and  average 
daily  simulated  prescription  fill  time;  February  is  an  extremely  volatile  month,  and  is 
removed  from  the  data  set.  First,  a  two-sided  runs  test  for  detecting  non-randomness  is 
performed,  and  the  results  indicate  there  is  no  evidence  to  suggest  dependence  in  the 
differences  over  time  (Bradley,  1968).  The  standardized  runs  test  statistic  is  -1.36  with  a 
p-value  of  0.17.  Next,  a  Durbin-Watson  test  is  conducted  to  detect  autocorrelation,  and 
the  results  are  consistent  with  the  two-sided  runs  test,  indicating  independence  in  the 
differences  over  time  (Durbin  &  Watson,  1950).  The  Durbin-Watson  test  statistic  is 
1.98  with  a  p-value  of  0.44.  The  difference  in  average  daily  prescription  fill  time  and 
average  daily  simulated  prescription  fill  time  does  not  show  any  evidence  of  non¬ 
randomness  or  autocorrelation. 
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C.  SUMMARY 

There  are  significant  trends  in  the  demand  and  prescription  fill  times  of 
prescriptions  filled  at  NMCSD  in  CY2014.  With  these  trends,  monthly  and  weekly 
indicators  are  used  to  generate  the  increased  manual  demand  that  is  expected  without  the 
automation;  this  simulated  manual  demand  is  combined  with  the  observed  manual 
demand  to  create  a  pre-implementation  data  set.  To  validate  this  simulation,  the 
automatic  workload  is  also  simulated  using  the  same  technique.  The  construction  of  the 
pre-automation  data  facilitates  analysis  of  the  effectiveness  of  the  automation 
implementation  at  NMCSD  conducted  in  Chapters  IV  and  V. 
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IV.  METHODOLOGY  AND  ANALYSIS 


The  objective  of  this  study  is  to  quantify  any  efficiencies  experienced  due  to  the 
implementation  of  pharmacy  automation  at  NMCSD.  In  addition,  linear  regression  is 
used  to  evaluate  which  characteristics  of  the  pre-automation  and  post-automation  data 
sets  most  heavily  impact  the  difference  in  average  prescription  fill  time.  The  results  of 
this  analysis  provide  a  foundation  for  the  implementation  of  pharmacy  automation  at 
similarly  sized  medical  facilities. 

A.  MANUAL  VS.  AUTOMATIC  FILLS  IN  CY2014 

Using  the  observed  post-automation  data  set,  the  prescription  fill  times  for  manual 
prescription  fills  and  automatic  prescription  fills  are  compared;  see  Figure  19.  In  order  to 
remove  any  evidence  of  autocorrelation,  the  observations  are  aggregated  by  week.  A  two- 
sided  runs  test  for  detecting  non-randomness  in  data  is  perfonned,  and  the  results  indicate 
there  is  no  evidence  to  suggest  the  data  points  were  produced  in  a  non-random  manner 
(standardized  runs  statistic  =  -0.58,  p-value  =  0.56).  On  average,  prescriptions  filled 
automatically  are  completed  6.97±0.97  (standard  error)  minutes  quicker  than 
prescriptions  filled  manually.  In  CY2014,  automatic  fills  were  faster  than  manual  fills  in 
96.2  percent  of  weeks,  with  an  approximate  95  percent  confidence  interval  for  the 
probability  that  the  automatic  fills  are  quicker  than  manual  fills  of  (0.87,  0.99).  This 
confidence  interval  does  not  include  0.5,  which  is  the  expected  probability  if  this  result 
occurred  by  chance;  therefore,  this  result  is  statistically  significant. 
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Figure  19.  Observed  Manual  and  Automatic  Fills  in  the  Post- Automation  Data 
Set.  The  average  weekly  prescription  fill  time  for  medications  filled 
automatically  tend  to  be  quicker  than  manually  in  96.2  percent  of 
weeks  in  CY2014. 

It  is  also  of  interest  to  see  if  there  is  a  significant  difference  in  the  average  fill 
time  during  periods  of  high  demand  in  the  pharmacy.  A  period  of  high  demand  is  defined 
as  an  hour  during  business  hours  (0800  to  2100  Monday  through  Friday  and  0800  to  1800 
Saturday  and  Sunday)  in  which  customer  demand  exceeds  the  mean  yearly  demand 
during  that  time  period;  the  mean  demand  on  weekdays  is  83  prescriptions  per  hour,  and 
the  mean  demand  on  weekends  is  34  prescriptions  per  hour.  These  observations  are 
aggregated  by  day,  and  the  average  daily  prescription  fill  times  during  hours  of  high 
demand  is  illustrated  in  Figure  20.  A  two-sided  runs  test  for  detecting  non-randomness  is 
performed,  and  the  results  indicate  there  is  no  evidence  to  suggest  dependence  in  the 
differences  over  time  (standardized  runs  statistic  =  -1.49,  p-value  =  0.14).  During  periods 
of  high  demand,  automatic  fills  were  quicker  than  manual  fills  on  84.5  percent  of  days 
with  an  approximate  95  percent  confidence  interval  for  the  probability  that  automatic  fills 
are  quicker  than  manual  fills  during  hours  of  high  demand  of  [0.80,  0.87].  This 
confidence  interval  does  not  include  0.5,  which  is  the  expected  probability  if  this  result 
occurred  by  chance;  therefore,  this  result  is  statistically  significant. 
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Figure  20.  Manual  and  Automatic  Fills  During  F[ours  of  High  Demand.  During 
periods  of  high  demand,  automatic  fills  tend  to  be  quicker  than 
manual  fills  84.5  percent  of  days  in  CY2014. 


B.  PRE- AUTOMATION  VS.  POST-AUTOMATION 

Using  both  the  pre-automation  data  set  generated  in  Chapter  III  and  the  observed 
post-automation  data  set,  the  average  daily  prescription  fill  times  pre-automation, 
Y Daily AvgPre  ’  are  compared  with  the  average  daily  prescription  fill  times  post-automation, 

Y Daily AxgPost  ’  see  Figure  21.  On  average,  prescriptions  filled  after  the  installation  of 

automation  are  completed  4.4±0.34  minutes  faster  than  prescriptions  filled  prior  to  the 
installation  of  automation.  The  average  daily  prescription  fill  time  post-automation  is 
faster  than  the  average  daily  prescription  fill  time  pre-automation  on  92.6  percent  of  day 
with  an  approximate  95  percent  confidence  interval  for  the  probability  that  post¬ 
automation  fills  are  quicker  than  pre-automation  fills  of  [0.89,  0.95].  This  confidence 
interval  does  not  include  0.5,  which  is  the  expected  probability  if  this  result  occurred  by 
chance;  therefore,  this  result  is  statistically  significant. 
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Figure  2 1 .  Average  Daily  Prescription  Fill  Time  Pre-Automation  and  Post- 

Automation.  The  average  daily  prescription  fill  time  for  medications 
filled  post-automation  is  quicker  than  medications  filled  pre¬ 
automation  on  92.6  percent  of  days. 

Next,  linear  regression  is  used  to  study  which  characteristics  of  the  data  most 
heavily  impact  the  difference  in  YDailyAvgPre  and  YDailyAvgPost.  This  difference  is  the  new 

response  variable,  YDjff,  defined  in  Equation  (5).  A  negative  value  of  YDjff  represents  a 

day  in  which  the  average  prescription  fill  time  pre-automation  is  quicker  than  post¬ 
automation. 


Y  -Y  -Y  659 

Equation  (6)  gives  the  linear  regression  model,  where  Y  is  the  response  variable, 
YDiff,  given  xl,...,xn  regressors;  the  constants,  J30  through  J3n,  are  the  unknown 

parameters,  and  s  is  the  error  term.  The  errors  are  modeled  as  independent  and  normally 
distributed  with  a  mean  of  zero  and  constant  variance  (Farraway,  2015).  Several  different 
characteristics,  including  daily  proportion  of  orders  automated,  proportion  of  highly 
prescribed  medications,  and  number  of  daily  high  demand  hours,  are  used  as  regressors. 
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In  this  analysis,  the  entire  month  of  February  is  eliminated  from  consideration  due  to  its 
relative  extremes. 


Y  -  fio  +  P\Xl  +  "•  +  PnXn  +  S 


(6) 


1.  Variable  Selection 

In  total,  ten  variables  are  considered  for  impact  on  YDiff :  total  number  of  hours  in 
the  day  above  the  mean  demand  (xHighDemand),  proportion  of  daily  demand  fill  by 
automation  in  the  post-automation  data  set  (xPropAutoPost),  proportion  of  medications 

demanded  in  the  top  live  most  prescribed  medications  pre-automation  and  post¬ 
automation  (xPropTop5MedsPre  and  xPropTop5MedsPost,  respectively),  proportion  of  medications 

demanded  in  the  top  ten  most  prescribed  medications  pre-automation  and  post¬ 
automation  {xPropTopI0MedsPre  and  xPropTopl0MedsPost,  respectively),  proportion  of  medications 

demanded  in  the  top  25  most  prescribed  medications  pre-automation  and  post-automation 
( x ProPToP25MedsPre  and  x ProPToP25MedsPost  >  respectively),  day  of  the  week  ( XD0W  ),  and  month  ( 
x Month ) *  Summary  statistics  for  the  ten  considered  variables  are  shown  in  Table  5. 
Variable  selection  is  performed  using  backwards  elimination;  the  full  model  with  all  ten 
variables  and  the  final  model  after  variable  selection  is  validated  using  regression 
diagnostic  techniques,  and  it  is  found  that  the  error  terms  can  be  modeled  as  independent 
with  mean  zero  and  constant  variance.  There  is  some  indication  that  the  errors  may  not 
be  normally  distributed,  but  with  the  large  number  of  observations,  inference  results  are 
not  affected  (Farraway,  2015).  In  addition,  none  of  the  outlying  observations  are  found  to 
be  influential.  Diagnostic  plots  can  be  found  in  Appendix  D.  Prior  to  the  discussion  of  the 
final  model  in  Section  2,  univariate  analysis  on  the  selected  variables  is  performed. 
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Table  5.  Summary  Statistics  for  Considered  Variables.  The  selected  model 
contains  three  continuous  variables.  The  coefficient  estimates  and 
regressor  summary  statistics  for  the  regressors  contained  in  the 
final  model  can  be  found  in  Table  6. 


Variable 

Final 

Model 

Type 

Minimum 

Value 

Maximum 

Value 

Mean 

Standard 

Deviation 

X 

HighDemand 

X 

Continuous 

0 

10 

6.8 

2 

^ Prop A  utoPost 

X 

Continuous 

0.11 

0.48 

0.38 

0.04 

^ Prop  Top5MedsPre 

X 

Continuous 

0.11 

0.29 

0.17 

0.03 

^ PropTop5MedsPost 

Continuous 

0.08 

0.28 

0.16 

0.03 

^ PropTopl  OMedsPre 

Continuous 

0.14 

0.35 

0.21 

0.04 

^ PropTopl  OMedsPost 

Continuous 

0.16 

0.41 

0.23 

0.05 

^  Prop  Top25MedsPre 

Continuous 

0.24 

0.49 

0.32 

0.05 

^  PropTopl  5MedsPost 

Continuous 

0.25 

0.53 

0.34 

0.05 

XDOW 

Categorical 

7  Levels 

N/A 

N/A 

^ Month 

Categorical 

1 1  Levels  (Excl.  Feb) 

N/A 

N/A 

a.  Hours  of  High  Demand 

As  defined  in  Section  A,  an  hour  of  high  demand  is  an  hour  during  business  hours 
in  which  the  prescription  demand  is  greater  than  the  mean  hourly  prescription  demand  for 
weekends  and  weekdays,  as  appropriate.  This  means  that  instead  of  customers  arriving  to 
the  pharmacy  at  a  relatively  steady  pace,  there  are  hours  of  peak  demand.  For  each  day, 
the  number  of  hours  the  pharmacy  functioned  in  high  demand  is  totaled,  and  this 
becomes  xHighDemand;  13.1  percent  of  days  experienced  8  or  more  hours  of  high  demand. 

Figure  22  reflects  a  slightly  decreasing  trend,  indicating  that  as  the  hours  of  high  demand 
increase,  the  difference  in  the  pre-automation  prescription  fill  time  and  the  post¬ 
automation  prescription  fill  time  decreases. 
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Figure  22.  Difference  in  Average  Daily  Prescription  Fill  Times  Based  Flours  of 
High  Demand.  As  the  number  of  hours  of  high  demand  increases,  the 
difference  in  average  daily  prescription  fill  time  between  pre¬ 
automation  and  post-automation  decreases. 


b.  Proportion  Filled  by  Automation 

On  average,  automation  filled  37.3  percent  of  prescriptions  in  CY2014;  however, 
there  are  days  in  which  automation  accounted  for  significantly  more  of  the  total  demand, 
47.7  percent,  or  significantly  less  than  the  total  demand,  11.3  percent.  In  addition, 
prescriptions  filled  automatically  have  a  mean  fill  time  that  is  6.94  minutes  faster  than 
prescriptions  filled  automatically.  The  proportion  of  post-automation  medications  filled 
through  automation,  xPwpAutoPost,  is  calculated  for  each  day.  The  linear  fit  of  xPropAutoPost 

shown  in  Figure  23  reflects  a  slightly  increasing  trend,  indicating  that  a  higher  proportion 
of  prescriptions  automated  causes  an  increase  in  YDiff. 
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Figure  23.  Difference  in  Average  Daily  Prescription  Fill  Times  Based  Daily 

Proportion  of  Prescriptions  Automated,  Post- Automation.  The  linear 
fit  reflects  a  slightly  increasing  trend,  indicating  that  a  higher 
proportion  of  prescriptions  automated  increases  the  difference  in 
average  daily  prescription  fill  time. 

c.  Frequently  Prescribed  Medications 

As  noted  in  Chapter  III,  the  type  of  medication  ordered  influences  the  prescription 
fill  time.  The  distribution  for  the  25  most  frequently  prescribed  medications  can  be  found 
in  Appendix  B.  These  medications  account  for  30.5  percent  of  total  prescriptions 
filled  pre-automation,  and  31.7  percent  of  total  prescriptions  filled  post-automation.  For 
both  the  pre-automation  and  post-automation  data,  the  daily  proportion  of  medications 
that  are  the  top  25,  ten,  and  five  most  prescribed  medications  is  found  to  create  the 
variables  xPmpTop5MedsPre,  xPropTop5MedsPost,  xPmpTopl0MedsPre,  xPwpTopl0MedsPost,  xPropTop25MedsPre,  and 

^ PropTop. i5MedsPost  •  These  variables  are  all  highly  correlated  with  each  other;  therefore,  it  is 

only  appropriate  to  select  one  of  them  for  use  in  the  linear  regression.  After  performing 
backwards  elimination,  xPropTop5MedsPre  is  selected  as  the  variable  with  the  most  impact  on 

YDiff.  This  is  a  reasonable  selection  because  frequently  prescribed  medications  are  stored 
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in  more  convenient  locations,  and  therefore,  their  fill  times  should  be  faster  than  less 
frequently  prescribed  medications.  When  all  prescriptions  are  filled  manually,  this 
convenience  could  make  a  large  difference  in  the  overall  daily  prescription  fill  time.  The 
linear  fit  of  xPropTop5MedsPre  shown  in  Figure  24  reflects  a  slightly  increasing  trend, 


indicating  that  a  higher  proportion  of  prescriptions  automated  causes  an  increase  in  Ynm . 
Linear  fits  of  x 


v  PropTop5MedsPos1  5  X PropToplOMedsPre’  X  Prop  Top  1  OMedsPost  5  X  PropTop25MedsPre 


DiJT 

,  and 


v  PropTop25MedsPost 


can  be  found  in  Appendix  E. 
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Figure  24.  Difference  in  Average  Daily  Prescription  Fill  Times  Based  Daily 
Proportion  of  Prescriptions  that  are  the  Five  Most  Prescribed 
Medications,  Pre-Automation.  The  linear  fit  reflects  a  slightly 
increasing  trend,  indicating  that  a  higher  proportion  of  highly 
prescribed  medications  increases  the  difference  in  average  daily 
prescription  fill  time. 


2.  Regression 

A  simple  linear  regression  is  fit  using  the  three  selected  continuous  variables: 
XnigkDemand’  x PropAutoPost  ’  and  x PropToP5Pre  ■  Equation  (7)  shows  the  resulting  model,  where 


each  estimate  of  the  coefficients  are  rounded  to  the  corresponding  standard  error.  A 
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positive  value  of  the  response  signifies  that  the  post-automation  prescription  fills  were, 
on  average,  faster  than  the  pre-automation  prescription  fills  on  that  particular  day.  A 
positive  (negative)  value  of  a  coefficient  indicates  that  as  the  variable  increases,  the 
difference  in  the  average  daily  prescription  fill  time  between  pre-automation  and  post¬ 
automation  becomes  more  positive  (negative).  Additional  details  for  the  regressors  can  be 
found  in  Table  6. 


-2x 


HoursHigh 


+  Sx 


Prop Auto  Post 


+  I2x 


PropTop5Pre 


(7) 


Table  6.  Model  Term  Coefficient  Estimates,  Standard  Error,  T-Ratio,  and 
Statistical  Significance.  Each  coefficient  is  rounded  in  the  final 
regression  model  according  to  the  corresponding  standard  error. 
Each  variable  is  significant  to  the  0.1  level. 


Term 

Estimate 

Standard 

Error 

T-Ratio 

P- Value 

Intercept 

0.55 

2.53 

-0.22 

0.8265 

^  HighDemand 

-0.25 

0.10 

-2.45 

0.0150 

X 

Prop  A  utoPost 

8.86 

4.79 

1.85 

0.0651 

^ Prop  Top5Pre 

12.30 

6.74 

1.83 

0.0688 

This  linear  regression  model  has  three  degrees  of  freedom  with  333  residual 
degrees  of  freedom.  These  values  correspond  to  an  F-ratio  of  6.870,  and  a  model  p-value 
of  0.0002;  this  result  indicates  there  is  a  significant  relationship  between  the  variables  in 

the  linear  regression  to  the  response.  The  R2adj  value,  which  is  a  measure  of  the  proportion 

of  the  variability  of  the  response  explained  by  the  regressors,  is  0.058  (Farraway,  2015). 
Though  this  is  an  extremely  low  value  for  this  measure,  it  is  not  inherently  limiting.  The 
regressors  are  still  statistically  significant,  and  conclusions  can  still  be  drawn  about  how 
changes  in  the  regressors  are  associated  with  changes  in  the  response.  The  intention  of 
this  study  is  not  to  predict  the  difference  in  average  daily  prescription  fill  times  for  pre¬ 
automation  and  post-automation,  but  to  determine  the  factors  that  impact  this  response. 
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The  impact  of  these  three  regressors  on  the  difference  in  average  daily 
prescription  fill  time  pre-automation  and  post-automation  can  be  inferred  from  the  values 
of  their  coefficients.  The  value  of  the  coefficient  for  xnighDemand  suggests  that  for  every 

increase  in  hours  of  high  demand  experienced  by  the  pharmacy,  the  estimated  expected 
response  decreases  by  0.2  minutes,  holding  the  values  of  the  other  two  regressors 
constant.  The  maximum  value  of  the  variable  found  in  the  data  is  ten  hours;  therefore, 
this  variable  could  potentially  decrease  the  response  by  two  minutes.  A  decrease  in  the 
response  indicates  that  the  implementation  of  automation  had  a  lesser  impact  on  the 
overall  average  daily  prescription  fill  time. 

The  proportion  of  prescriptions  filled  through  automation,  xPropAutoPost,  causes  an 

increase  in  response.  This  variable  can  take  on  values  between  zero  and  one,  and  the 
maximum  value  of  ,  observed  in  the  data  set  is  0.48;  this  indicates  that 

Prop AutoPost  5 

x Prop AutoPost  could  potentially  increase  the  response  by  3.84  minutes,  holding  the  values  of 

the  other  two  regressors  constant.  An  increase  in  the  response  indicates  that  the 
implementation  of  automation  has  a  greater  impact  on  the  overall  daily  prescription  fill 
time.  Figure  25  depicts  the  change  in  the  response  for  different  values  of  xHighDemand  for 

fixed  values  of  xPropAutoPost. 
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Figure  25.  Estimated  Expected  Difference  in  Average  Daily  Prescription  Fill 
Time  as  Hours  of  High  Demand  Increases  for  Fixed  Values  of 
Proportion  of  Prescriptions  Automated.  The  proportion  of  automated 
prescription  fills  is  fixed  at  the  first  quartile  and  maximum  values, 
while  the  hours  of  higher  than  average  demand  are  varied.  As  the 
hours  of  higher  than  average  demand  increase,  the  estimated 
expected  difference  in  average  daily  prescription  fill  time  decreases. 
The  dotted  lines  represent  95  percent  confidence  intervals. 


The  proportion  of  highly  prescribed  medications  pre-automation,  xPropTop5Pre ,  also 
positively  impacts  YDiff.  This  variable  takes  on  values  between  zero  and  one  and  the 
maximum  value  of  xD  T  observed  in  the  data  set  is  0.29.  Based  on  this  information, 
an  increase  in  xPropTop5Pre  could  potentially  increase  the  difference  in  average  daily 

prescription  fill  times  by  3.48  minutes,  holding  the  values  of  the  other  two  regressors 
constant.  This  increase  is  an  interesting  result,  because  frequently  ordered  medications 
tend  to  be  stored  in  more  convenient  locations;  it  is  expected  that  this  would  cause  a 
decrease  in  YDiff .  The  observed  result,  an  increase,  could  be  explained  by  an  optimized 

loading  of  medications  into  the  automation  machine.  Figure  26  depicts  the  change  in  the 
response  for  different  values  of  xHighDemand  or  fixed  values  of  xPropTop5Pre . 
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Figure  26.  Estimated  Expected  Difference  in  Average  Daily  Prescription  Fill 
Time  as  Flours  of  High  Demand  Increases  for  Fixed  Values  of 
Proportion  of  Frequently  Prescribed  Medications.  The  proportion  of 
frequently  prescribed  medications  is  fixed  at  the  first  quartile  and 
maximum  values,  while  the  hours  of  higher  than  average  demand  are 
varied.  As  the  hours  of  higher  than  average  demand  increase,  the 
estimated  expected  difference  in  average  daily  prescription  fill  time 
decreases.  The  dotted  lines  represent  95  percent  confidence  intervals. 


C.  SUMMARY 

Through  analysis  of  the  data,  it  is  revealed  that  the  difference  in  average  daily 
prescription  fill  times  is  statistically  significant,  both  between  manual  and  automatic  fills 
in  CY2014,  and  between  pre-automation  and  post  automation.  In  addition,  linear 
regression  reveals  that  there  are  three  main  factors  that  affect  the  difference  in  average 
daily  prescription  fill  times:  the  number  of  hours  of  high  demand  experienced  within  the 
pharmacy,  the  proportion  of  prescriptions  filled  by  automation,  and  the  proportion  of 
highly  prescribed  medications  demanded  by  customers.  As  the  number  of  hours  of  high 
demand  increases,  the  response  decreases,  while  as  the  two  proportions  increase,  the 
response  also  increases. 
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V.  RESULTS,  CONCLUSIONS,  AND  RECOMMENDATIONS 


This  thesis  demonstrates  a  method  to  evaluate  the  changes  in  efficiency 
experienced  at  the  NMCSD  pharmacy  after  the  implementation  of  phannacy  automation. 
The  introduction  of  barcode  scanning  for  each  step  in  the  prescription  fill  process 
provides  a  unique  opportunity  to  objectively  evaluate  this  process.  With  the  increase  in 
emerging  technologies  now  available  throughout  different  areas  of  healthcare,  it  is  in  the 
Navy’s  best  interest  to  invest  in  items  that  could  potentially  lead  to  better  access  to  care; 
this,  in  turn,  leads  to  healthier  Sailors  and  Marines. 

A.  POST-AUTOMATION  TRENDS 

Several  items  of  interest  emerged  during  exploratory  analysis  of  the  post¬ 
automation  data  set.  With  more  than  one  year’s  worth  of  data,  these  trends  could  be 
further  studied  to  determine  if  they  are  reoccurring,  or  specifically  related  to  events  in 
CY2014.  The  most  significant  and  problematic  item  is  the  sharp  increase  experienced  in 
the  month  of  February.  During  this  month,  the  average  prescription  fill  time  during 
working  hours  on  weekdays  is  63.3  ±  162.2  (standard  deviation)  minutes.  In  comparison, 
the  average  prescription  fill  time  for  the  rest  of  the  year  during  this  time  same  time  period 
is  24.3  ±55.9  minutes.  The  month  of  February  also  experiences  an  increase  in  average 
daily  demand  of  197  prescriptions  compared  to  every  other  month  in  the  year.  It  can  be 
hypothesized  that  this  departure  from  the  norm  is  associated  with  the  cold  and  flu  season 
and  an  increase  in  patient  visits  to  NMCSD;  if  is  it  determined  that  this  increase  in 
demand  and  prescription  fill  time  is  not  an  anomaly,  extra  pharmacists  and  phannacy 
technicians  should  be  staffed  during  this  month  to  account  for  the  increase  in  demand. 

Next,  data  is  aggregated  by  day  of  the  week.  There  is  an  obvious  difference  in 
both  demand  and  average  prescription  fill  time  between  weekdays  and  weekends;  this  is 
expected  based  on  the  NMCSD ’s  primarily  military  patient  population.  A  more 
interesting  observation  is  the  average  prescription  fill  time  is  significantly  higher  on 
Mondays  than  every  other  day  of  the  week,  but  that  there  is  no  corresponding  increase  in 
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demand.  During  working  hours,  the  average  time  to  fill  a  prescription  on  Mondays  is 
34. 9±  108.1  minutes,  compared  to  every  other  weekday,  which  is  26.1  ±61.6  minutes 

Again,  much  of  this  increase  can  be  attributed  to  the  volatile  month  of  February. 
Excluding  February,  the  average  hourly  prescription  fill  time  during  working  hours  on 
Mondays  is  26.8  ±66.4  minutes,  while  in  February  alone,  the  average  hourly  prescription 
fill  time  on  Mondays  is  114.6 ±275.5  minutes.  Figure  27  compares  the  hourly 
prescription  fill  times  in  February  to  the  rest  of  the  year.  While  the  times  are  considerably 
higher  during  February,  there  is  still  a  substantial  difference  in  the  fill  times  from  1 800— 
1900  during  the  rest  of  the  year.  Excluding  February,  the  average  prescription  fill  time 
during  this  hour  on  Mondays  is  8 1.6  ±267. 9  minutes.  This  increase  in  prescription  fill 
time  during  1800-1900  is  not  related  to  an  increase  in  demand;  the  average  demand 
during  this  hour  is  40  prescriptions,  compared  to  average  of  70  items  per  hour  during  the 
rest  of  the  day.  Prescription  fill  time  only  includes  the  time  it  takes  to  get  the  prescription 
to  the  pharmacist  for  verification,  which  means  a  customer  could  potentially  be  waiting 
upwards  of  90  minutes  for  their  specific  order. 


Figure  27.  Average  Hourly  Prescription  Fill  Time  During  Working  Hours, 

Monday,  February  Only  and  Excluding  February.  The  black  bars 
represent  standard  error  of  the  mean.  Though  Mondays  in  February 
represent  a  considerable  departure  from  the  average,  the  average 
prescription  fill  time  in  the  hour  from  1800  to  1900  is  significantly 
higher  than  the  rest  of  the  workday,  without  a  corresponding  increase 
in  demand. 
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This  increase  in  prescription  fill  time  during  the  latter  part  of  the  day  is  not  just 
limited  to  Mondays,  however.  If  the  entire  year  is  observed,  there  is  an  increase  in 
average  prescription  fill  time  during  the  hour  from  1800  to  1900,  but  the  demand  is 
almost  at  a  daily  low.  This  increase  in  prescription  fill  time  is  also  not  attributable  to 
variance  in  manual  fill  times;  Figure  28  displays  the  average  prescription  fill  time, 
excluding  February,  for  weekdays  between  0700  and  2200,  separated  by  type  of  fill. 
Average  manual  fill  times  from  1800  to  1900  are  34.4 ±98.8  with  a  corresponding 
average  hourly  demand  of  only  18  items,  while  average  automatic  fill  times  during  the 
same  hour  are  29.9  ±79.8  minutes  with  a  corresponding  average  hourly  demand  of  13 
items.  In  comparison,  for  the  rest  of  the  working  day,  an  average  manual  fill  takes  26.7  ± 
55.2  minutes  with  an  average  demand  of  37  prescriptions;  an  average  automatic  fill  is 
completed  in  19.3  ±39.8  minutes  with  an  average  demand  of  21.3  prescriptions. 


Figure  28.  Average  Hourly  Prescription  Demand  and  Prescription  Fill  Time, 
Working  Hours,  Monday  through  Friday,  excluding  February.  The 
black  bars  represent  standard  error  of  the  mean.  Though  the  demand 
and  average  prescription  fill  time  for  manual  fills  is  higher  than 
automatic  fills  for  all  hours  during  the  work  week,  both  manual  and 
automatic  fills  experience  a  delay  from  1800  to  1900  without  a 
corresponding  increase  in  prescription  demand. 
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Without  more  information,  it  is  difficult  to  determine  the  source  of  these  delays. 
The  absence  of  a  corresponding  increase  in  demand  indicates  this  increase  in  prescription 
fill  time  in  unrelated  to  an  influx  of  customers  into  the  pharmacy.  Both  the  manual 
prescription  fill  times  and  the  automatic  prescription  fill  times  increase  significantly 
during  this  hour;  there  is  a  28.8  percent  increase  in  average  hourly  prescription  fill  times 
for  manual  fills  and  a  51.8  percent  increase  for  automatic  fills.  This  evidence  suggests  the 
delays  are  related  to  workflow  management  within  the  pharmacy. 

B.  PRE-AUTOMATION  COMPARED  TO  POST-AUTOMATION 

When  evaluating  the  total  time  spent  filling  prescriptions  both  pre-automation  and 
post-automation,  it  is  important  to  understand  that  several  technicians  are  filling 
prescriptions  at  the  same  time;  in  the  post-automation  environment,  the  automation 
system  is  also  continuously  routing  prescriptions  through  the  fill  process.  The  total  time 
spent  filling  prescriptions  reflects  the  amount  of  time  it  would  take  to  fill  each 
prescription  sequentially,  one  at  a  time.  In  most  instances,  it  is  most  appropriate  to  refer 
to  total  prescription  fill  time  for  the  year  in  the  units  of  days. 

During  regression  analysis,  one  of  the  significant  indicators  of  a  larger  difference 
between  the  average  daily  prescription  fill  time  pre-automation  and  post-automation  is 
the  post-automation  proportion  of  prescriptions  filled  automatically.  Overall,  the 
prescriptions  filled  automatically  account  for  37.3  percent  of  total  prescription  fills  in 
CY2014,  with  a  maximum  of  47.7  percent  and  a  minimum  of  11.2  percent.  With  all 
other  factors  held  constant,  an  average  automation  proportion  of  0.37  is  related  to  an 
increase  in  the  difference  of  average  prescription  fill  time  of  2.98  ±  1.95  (standard  error) 
minutes;  if  this  proportion  can  be  maintained  over  the  course  of  an  entire  year,  assuming 
an  average  daily  weekday  demand  is  1186  prescriptions,  this  could  reduce  yearly 
workload  by  an  average  of  636  days  alone,  compared  to  pre-automation.  Figure  29 
illustrates  the  yearly  reduction  in  weekday  workload  based  on  proportion  of  prescriptions 
filled  through  automation. 
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Figure  29.  Estimated  Yearly  Reduction  in  Weekday  Workload  Based  on 
Proportion  of  Prescriptions  Filled  Through  Automation  and 
Demand.  If  the  proportion  of  prescriptions  automated  can  be 
maintained  at  0.37,  there  is  potential  to  reduce  weekday  workload 
within  the  pharmacy  by  636  days,  compared  to  pre-automation. 

With  all  other  regressors  held  constant,  there  is  no  additional 
reduction  of  work  in  any  scenario  when  the  proportion  of  workload 
automated  is  set  to  zero,  causing  a  non-linear  line. 

This  same  type  of  logic  can  be  applied  to  the  daily  workload  of  each  pharmacy 
technician  in  NMCSD.  Based  on  the  January  2015  schedule,  at  a  minimum,  25  pharmacy 
technicians  are  assigned  to  work  in  the  pharmacy  from  0745  to  1615  every  weekday,  with 
other  additional  staff  during  various  parts  of  the  workday;  less  pharmacy  technicians  are 
assigned  during  non- working  hours  (B.  Detlef,  BUMED,  personal  communication, 
August  12,  2015).  Assuming  there  are  25  pharmacy  technicians  working  within  the 
pharmacy,  if  the  proportion  of  prescriptions  automated  is  held  constant  at  0.37  for  an 
entire  day,  with  an  average  weekday  demand  of  1,186  items,  the  daily  workload  for  each 
technician  can  be  reduced  by  2.34  ±0.03  hours,  compared  to  pre-automation.  When  this 
proportion  is  maintained  at  0.50,  the  daily  workload  can  be  reduced  by  3. 16  ±0.02  hours, 

which  is  a  0.40  Full  Time  Equivalent  (FTE),  assuming  an  eight-hour  workday.  Figure  30 
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illustrates  this  reduction  in  daily  workload  based  on  proportion  of  prescriptions 
automated  and  number  of  pharmacy  technicians  working  in  the  pharmacy. 


Figure  30.  Estimated  Daily  Reduction  in  Weekday  Workload  Based  on 

Proportion  of  Prescriptions  Filled  Through  Automation  and  Number 
of  Pharmacy  Technicians.  If  the  proportion  of  prescriptions 
automated  can  be  maintained  at  0.37,  with  ten  pharmacy  technicians 
working  and  an  average  demand  of  1,189  items,  there  potential  to 
reduce  daily  workload  by  2.34  hours,  which  is  the  same  as  0.29 
FTEs.  With  all  other  regressors  held  constant,  there  is  no  additional 
reduction  of  work  in  any  scenario  when  the  proportion  of  workload 
automated  is  set  to  zero,  causing  a  non-linear  line. 


It  is  also  important  to  consider  the  implementation  from  a  cost  perspective.  The 
pharmacy  automation  system  at  NMSCD  carried  an  individual  implementation  cost  of 
$2.4  million  of  the  total  $49  million  contract.  (B.  Detlef,  BUMED,  personal 
communication,  September  10,  2014).  Based  on  the  watchbill  from  January  2015,  most 
of  the  pharmacy  technicians  are  of  the  rank  Petty  Officer  Third  Class  (E-4),  so  the  base 
salary  for  this  rank,  with  over  three  years  of  service,  will  be  used  at  the  basis  for  cost 
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analysis  (B.  Detlef,  BUMED,  personal  communication,  August  12,  2014);  there  is  a  mix 
of  several  military  ranks,  as  well  as  civilians  and  contractors,  which  suggests  that  this 
estimate  is  conservative.  Appendix  F  details  the  conversion  of  E-4  military  salary  to 
hourly  wage. 

When  directly  comparing  the  total  time  spent  filling  prescriptions  pre-automation 
and  post-automation,  recall  the  yearly  demand  for  each  is  roughly  the  same:  364,463  and 
364,531  observations,  respectively.  Pre-automation,  the  total  time  spent  filling 
prescriptions  is  7,699  days,  while  the  total  time  spent  filling  prescriptions  post 
automation  is  6,604  days;  this  is  a  reduction  of  14.2  percent.  This  reduction  in  time  spent 
filling  prescriptions  is  associated  with  $330,000  reduction  in  cost,  the  amount  of  money  it 
would  cost  to  pay  for  the  equivalent  amount  of  work  from  E-4  military  members 
established  only  on  base  pay.  With  these  calculations,  it  would  take  roughly  seven  years 
to  recoup  the  cost  of  implementation  at  NMCSD  based  on  increased  efficiency. 

These  are  extremely  conservative  estimates.  In  reality,  many  other  factors 
contribute  to  the  benefits  of  implementing  a  new  system.  For  example,  the  increased 
tracking  ability  due  to  the  barcode  scanning  of  medications  could  potentially  lead  to 
increased  efficiency  in  the  stocking  and  ordering  of  medications.  In  addition,  wait  times 
in  the  pharmacy  are  a  major  source  of  complaints  for  many  customers  within  a  medical 
facility,  and  the  main  reason  many  customers  choose  to  have  prescriptions  filled  at  non¬ 
military  phannacies.  Prescriptions  filled  at  non-military  pharmacies  are  more  expensive 
for  the  Navy  to  fill,  and  the  potential  decrease  in  customer  wait  time  within  the  pharmacy 
could  then  be  tied  to  a  recapture  of  costs  from  non-military  pharmacy  prescriiption  fills. 
Decreased  wait  time  also  likely  leads  to  better  patient  satisfaction,  which  is  an  intangible 
benefit  of  such  an  upgrade. 

C.  CONCLUSIONS  AND  FUTURE  WORK 

After  the  implementation  of  pharmacy  automation  at  NMCSD,  there  is  an 
increase  in  efficiency,  defined  as  the  total  prescription  fill  time.  This  efficiency  can  be 
quantified  as  number  of  reduction  total  time  to  fill  a  prescription,  reduction  in  FTEs,  or 
reduction  in  equivalent  salary  of  pharmacy  technicians.  This  data  is  collected  after  a  little 
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more  than  one  year  of  implementation;  with  better  training  and  more  emphasis  placed  on 
the  optimal  automation  workload,  these  savings  in  efficiency  will  continue  to  increase. 

There  is  great  potential  for  future  study  into  this  subject.  With  the 
implementation  of  the  barcode  scanning  system,  more  in-depth  investigations  can  be 
made  into  the  optimal  ordering  and  stocking  of  frequently  filled  medications.  In  addition, 
this  study  does  not  attempt  to  quantify  the  impacts  of  the  automation  system  on 
prescription  accuracy,  patient  satisfaction,  or  overall  access  to  care.  With  additional  data, 
comparisons  can  be  made  between  changes  in  the  pharmacy  to  increases  or  decreases  in 
these  areas  with  all  of  NMCSD.  Most  importantly,  this  study  is  limited  by  lack  of  data, 
both  in  duration  of  post-automation  data  collection  and  lack  of  pre-automation  data;  with 
additional  post-automation  data,  a  more  significant  evaluation  of  possible  trends  observed 
within  the  phannacy  could  be  conducted,  and  with  pre-automation  data,  a  considerably 
better  comparison  could  be  completed. 
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APPENDIX  A.  PROCESSES  TRACKED  IN  PRESCRIPTION  DATA 


Table  7.  Processes  Tracked  in  Prescription  Data.  The  prescription  data 
contains  the  date  and  time,  accurate  to  the  second,  of  fifteen 
specific  processes  during  the  entire  prescription  order  procedure. 
Priority  High,  Priority  Urgent,  Initial  Pending,  Suspend, 

Reschedule  to  Manual,  and  Rescheduled  to  Automatic  do  not  occur 
with  every  prescription;  these  processes  tend  to  happen  when  there 
is  a  complication.  Only  the  eight  processes  outline  in  Chapter  III 
are  specifically  relevant  to  this  study. 


Process 

Description  (date/time) 

Scheduled 

Entered  into  the  system 

Diverted 

Divert  from  an  automatic  fill  to  manual  fill,  reason  provided 

Priority  High 

Priority  is  updated  from  standard  to  high 

Priority  Urgent 

Priority  is  updated  from  high  to  urgent 

Initial  Pending 

Initially  put  into  a  “pending”  status,  reason  provided 

Suspend 

Suspended  from  completion,  reason  provided 

Rescheduled  to  Manual 

Rescheduled  from  an  automatic  fill  to  manual  fill 

Rescheduled  to  Automatic 

Rescheduled  from  manual  fill  to  automatic  fill 

Counted 

Count  of  quantity  of  medication  is  complete 

Added  to  Tote 

Counting,  bottling,  capping,  and  labeling  is  complete 

Verified 

Phannacist  verification  for  errors  is  complete 

Added  to  Bag 

Combined  with  remaining  customer  order 

Added  to  Location 

Complete  order  is  placed  in  assigned  cubby  location 

Ready  for  Pickup 

Order  is  complete  and  ready  for  customer  pick  up 

Pickup 

Customer  picks  up  completed  order. 
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APPENDIX  B.  FREQUENCY  OF  PRESCRIBED  MEDICATIONS 
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Figure  3 1 .  Most  Frequently  Prescribed  Medications,  Post  Automation.  The  25 
most  frequently  prescribed  medications  comprise  31.7  percent  of  all 
post-automation  prescription  fills. 
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Figure  32.  Most  Frequently  Prescribed  Medications,  Pre-Automation.  The  25 
most  frequently  prescribed  medications  comprise  30.5  percent  of  all 
pre-automation  prescription  fills 
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APPENDIX  C.  DIFFERENCE  IN  AVERAGE  FILL  TIME  IN 
OBSERVED  AND  SIMULATED  DATA  SET,  MONTH  BY  DAY  OF 

THE  WEEK. 


Figure  33.  Month  by  Day  of  the  Week  for  the  Difference  in  Average  Daily 
Prescription  Fill  Time  between  the  Observed  and  Simulated 
Automation  Data  Set,  First  Quadrimester  CY2014.  Other  than  the 
month  of  February,  there  does  not  appear  to  be  any  significant 
patterns  based  on  month  or  day  of  week 
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Figure  34.  Month  by  Day  of  the  Week  for  the  Difference  in  Average  Daily 
Prescription  Fill  Time  between  the  Observed  and  Simulated 
Automation  Data  Set,  Second  Quadrimester  CY2014.  There  does  not 
appear  to  be  any  significant  patterns  based  on  month  or  day  of  week 


62 


Figure  35.  Month  by  Day  of  the  Week  for  the  Difference  in  Average  Daily 
Prescription  Fill  Time  between  the  Observed  and  Simulated 
Automation  Data  Set,  Third  Quadrimester  CY2014.  There  does  not 
appear  to  be  any  significant  patterns  based  on  month  or  day  of  week 
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APPENDIX  D.  REGRESSION  DIAGNOSTICS 
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Figure  36.  Regression  Diagnostic  Plots  for  Final  Linear  Regression  Model.  The 
final  model  is  validated  using  regression  diagnostics.  The  four 
residual  plots  indicate  that  the  error  terms  have  constant  variance  and 
mean  zero,  but  may  not  be  normally  distributed.  In  addition,  none  of 
the  outlying  observations  are  found  to  be  influential. 
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APPENDIX  E.  LINEAR  FITS  OF  ELIMINATED  VARIABLES 
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Figure  37.  Difference  in  Average  Daily  Prescription  Fill  Times  Based  on  Day 
of  the  Week.  The  linear  fit  reflects  a  straight  line,  indicating  that  day 
of  the  week  does  not  impact  the  difference  in  average  daily 
prescription  fill  time. 
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Figure  38.  Difference  in  Average  Daily  Prescription  Fill  Times  Based  on 

Month.  The  linear  fit  reflects  a  straight  line,  indicating  that  month 
does  not  impact  the  difference  in  average  daily  prescription  fill  time. 
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Figure  39.  Difference  in  Average  Daily  Prescription  Fill  Times  Based  Daily 
Proportion  of  Prescriptions  that  are  the  Five  Most  Prescribed 
Medications,  Post-Automation.  The  linear  fit  reflects  a  slightly 
increasing  trend,  indicating  that  a  higher  proportion  of  highly 
prescribed  medications  increases  the  difference  in  average  daily 
prescription  fill  time. 
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Figure  40.  Difference  in  Average  Daily  Prescription  Fill  Times  Based  Daily 
Proportion  of  Prescriptions  that  are  the  Ten  Most  Prescribed 
Medications,  Pre-Automation.  The  linear  fit  reflects  a  slightly 
increasing  trend,  indicating  that  a  higher  proportion  of  highly 
prescribed  medications  increases  the  difference  in  average  daily 
prescription  fill  time. 
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Figure  4 1 .  Difference  in  Average  Daily  Prescription  Fill  Times  Based  Daily 
Proportion  of  Prescriptions  that  are  the  Ten  Most  Prescribed 
Medications,  Post-Automation.  The  linear  fit  reflects  a  slightly 
increasing  trend,  indicating  that  a  higher  proportion  of  highly 
prescribed  medications  increases  the  difference  in  average  daily 
prescription  fill  time. 
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Proportion  of  Daily  Total  in  Top  25  Most  Frequently  Prescribed  Medications 

Pre-Automation 


Figure  42.  Difference  in  Average  Daily  Prescription  Fill  Times  Based  Daily 
Proportion  of  Prescriptions  that  are  the  25  Most  Prescribed 
Medications,  Pre-Automation.  The  linear  fit  reflects  a  slightly 
increasing  trend,  indicating  that  a  higher  proportion  of  highly 
prescribed  medications  increases  the  difference  in  average  daily 
prescription  fill  time. 
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Figure  43.  Difference  in  Average  Daily  Prescription  Fill  Times  Based  Daily 
Proportion  of  Prescriptions  that  are  the  25  Most  Prescribed 
Medications,  Post-Automation.  The  linear  fit  reflects  a  slightly 
increasing  trend,  indicating  that  a  higher  proportion  of  highly 
prescribed  medications  increases  the  difference  in  average  daily 
prescription  fill  time. 
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APPENDIX  F.  CONVERSION  OF  MONTHLY  EARNINGS  TO 

HOURLY  WAGES 


Military  salary  is  normally  presented  in  terms  of  monthly  earnings.  Equation  (8) 
details  the  conversion  of  monthly  earnings  for  an  E-4  into  hourly  wages  (Defense 
Finance  and  Account  Service,  2015). 


$2238.07 
Month  / 


12  Months  f  1  Year  ' 
1  Year  / 1 261  Weekdays^ 


1  Weekday  ^  f  $12.86^ 
8  Work-Hours/  v  Hour  / 


(8) 
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